Controlling hardware settings via MIDI - Part 1 of 2
System Exclusive messages are used by synthesizers to allow programming of devices to be done over a midi connection. In the case of the EWI USB it is used to set the various controller states and data outputs. Anything you set in the "EWI Configuration" page of the Aria Software is sent to the controller via system exclusive.
So why would you want to know how to program it? The Aria software is not the only soft synth or device you might use with the EWI USB, but it is the only program they provided to let you configure it. To use the Aria software to configure it the software has to take over both the input (to read the current config) and output (to send a new config) midi ports, meaning that while Aria is configuring the device you cannot use your other synth... This means you can't set the internal transposition (meaning you have to rely on your client synth if you want to change your key offset), change the dynamic range, or change what controllers the unit is outputting. While you can load and send data to the EWI from the Aria configuration panel, it would be nice to have a standalone application that takes a bit less in resources, or at least the information to make controller plugins for the various other softwares you might like to use... or to be able to write your own software for controlling the device.
Before I went and spent hours or even days working on this, I asked Akai for this information, and they said they didn't know and I should ask Garritan - a scary response since they are the people who built the bloody thing?!? Does this mean the firmware inside the EWI USB and therin everything output by it was ALSO done by Garritan? Garritan has yet to respond by either e-mail or their online flash based contact form after three attempts and some odd month later, So reverse engineering it is.
Which oddly enough could work out for the better since if I clean room reverse engineering the sysex, one could make a cloned firmware for something like the arduino to allow the construction of your own ARIA software compatable device without legal issues. (...and I've wanted to build a electric wind instrument from scratch for two decades)
One nice thing the Aria software does do is let you save your controller information as .syx files, which upon inspection in a hex editor reveals they are in fact just raw sysex data - the 'setup controls' for example looks something like this in hex:
F0 47 7F 6D : 00 00 06 40 : 40 40 40 07 : 7F F7
14 bytes, F0 is the standard start of sysex, while F7 is the end. While the other two tabs, 'performance controls' and 'controller config' seem to share this 19 byte long sysex:
F0 47 7F 6D : 02 00 0B 00 : 00 4A 20 02 : 00 00 7F 00 : 00 00 F7
Some testing reveals that you can take these sysex and send them from any other midi program to change the settings. This means you could save from the aria file, load them into other programs and use them in realtime - it also gives us an easy means to figure out what all the different values are - just make a change, save the file, and look at the changes in the file!
But when you start the aria software's "EWI Config" panel it somehow reads the current settings on the device, so it must be sending a sysex read... So how do we figure out how to read the settings from the device.
My approach was to make use of 'midi looping' driver, specifically loopbe30 - a simple system device driver for windows that when you send it midi data on it's emulated 'input' it just echo's it back out on it's 'output'. LoopBe30 provides 30 of these emulated MIDI I/O ports and is a really handy tool for diagnosing midi conflicts or to hook multiple applications together. They have a free version, LoopBe1 that provides just a single channel. I was able to use Midi-OX to link the outputs/inputs from one device to another while monitoring all traffic going through.
Basically, I set the aria software to output on the first loopBE port, and to recieve input on the second loopBe Port. I then set up Midi-Ox to recieve data from the first loopBe port, display all data recieved on the monitor window, then echo that data to the EWI USB. For data coming from the EWI I just reverse that - Have Midi-Ox display all inputs from the EWI, then echo it to the second loopbe port so Aria recieves it.
Once I had that convoluted routing in place, I was able to snoop on what was going on in there. When you start the aria "EWi config" page, it sends a sysex message, the EWI responds back in kind, a second sysex is sent, the ewi responds again.
Send: TIMESTAMP IN PORT STATUS DATA1 DATA2 CHAN NOTE EVENT 00010012 3 5 F0 Buffer: 8 Bytes System Exclusive SYSX: F0 47 7F 6D : 42 00 00 F7 Responds: TIMESTAMP IN PORT STATUS DATA1 DATA2 CHAN NOTE EVENT 00001084 2 6 F0 Buffer: 19 Bytes System Exclusive SYSX: F0 47 00 6D : 02 00 0B 00 : 00 4A 20 02 : 00 00 7F 00 : 00 00 F7 Send: TIMESTAMP IN PORT STATUS DATA1 DATA2 CHAN NOTE EVENT 00010012 3 5 F0 Buffer: 8 Bytes System Exclusive SYSX: F0 47 7F 6D : 40 00 00 F7 Responds: TIMESTAMP IN PORT STATUS DATA1 DATA2 CHAN NOTE EVENT 0000108A 2 6 F0 Buffer: 14 Bytes System Exclusive SYSX: F0 47 00 6D : 00 00 06 40 : 40 40 40 07 : 7F F7
Those responses from the EWI look damned familiar, do they not? I was EXPECTING a dump surrounded by the normal sysex dump indicators, instead it sends the exact same sysex it should recieve without the extra formatting - making our lives even simpler. It also appears that before and after each sysex some extra NRPN data is being sent - specifically:
Before each sysex are sent: TIMESTAMP IN PORT STATUS DATA1 DATA2 CHAN NOTE EVENT 000CD1CE 3 5 B0 63 01 1 --- CC: NRPN MSB 000CD1CE 3 5 B0 62 04 1 --- CC: NRPN LSB 000CD1CE 3 5 B0 06 20 1 --- CC: Data Entry MSB After all sysex are sent: TIMESTAMP IN PORT STATUS DATA1 DATA2 CHAN NOTE EVENT 000CD1D8 3 5 B0 63 01 1 --- CC: NRPN MSB 000CD1D8 3 5 B0 62 04 1 --- CC: NRPN LSB 000CD1D8 3 5 B0 06 10 1 --- CC: Data Entry MSB
I'm not 100% certain what those are doing, though if I was to take a wild guess, I would assume the first one is some sort of "all notes off, stop sending midi data" as continuing to try and output values while sysex is going on would make the software have to be a lot more complex, and it's probably simpler to just put everything 'on hold'. The latter one is probably the reverse, a 'resume normal operation' command... At least, that's how I'd handle it. If I were to use the syx files to send data to the EWI from inside something like Sonar or Numerology, I'd make sure I send those NRPN values before and after just to be on the safe side.
Digging deeper the only difference between the sysex sent from the controller when requested as a dump and the save files appears to be that if byte three is 00 it is information read from the device, while changing byte 3 to 7F tells the EWI that the information is to be written. We can test this theory by looking at what happens if we say... change breath gain from 64 (40 hex) to 65 (41 hex). If all the assumptions so far are correct, that should send something much like the values in our save file for that setting:
F0 47 7F 6D : 00 00 06 41 : 40 40 40 07 : 7F F7
Interestingly it does not do that at all - instead it sends more NRPN:
TIMESTAMP IN PORT STATUS DATA1 DATA2 CHAN NOTE EVENT 00034A99 3 5 B0 63 00 1 --- CC: NRPN MSB 00034A99 3 5 B0 62 00 1 --- CC: NRPN LSB 00034A99 3 5 B0 06 41 1 --- CC: Data Entry MSB
This could be really handy for changing values the gain values on the fly if playing along with midi. This makes a good deal of sense as why send a 14 byte sysex when a six byte NRPN will do the job of changing just one controller? This leaves the sysex to be used for mass changes and settings saves, not the realtime change of a single value. It appears trying some of the other controllers that the least significant byte (LSB) controls which of the values on the EWI to change - 00 being breath gain, 01 being bite gain, 02 being bite AC gain, etc, etc down the list.
This means in addition to sysex, we also will want to document NRPN values sent to the EWI... Which we find out are in identical order in part 2.