i've started writing my datastream layout discovery engine. it's getting there in theory i just have to work some design issues out...

the idea is we have a memory topology definition of the entire t-side and e-side, with variables defaulting to RAW. i think i'll build my own definition format, it'll be pretty simple. the memory definition itself will be 'load once on startup' and not be runtime configurable.

i'm still a bit torn on how i want to do the calculations. i think a math engine like tunerpro has would introduce too much overhead. i might force people to use a set of hard-coded defined formulas, and if a new formula is required, you can ask.

there will be a concept of 'view levels' built right into the memory map, so we can default the ui to 'view essentials' instead of 'view everything', you wont see the 'raw' stuff unless you ask for it. view levels will be useable in graphing and stuff too so you can graph any raw memory address you want.

it'll do a discovery of the t-side on initial connect (and e-side if available) grabbing the entire datastream configuration, then build a datastream_topology which is basically just a list of pointers to definitions of memory addresses in that particular message. the raw datastream configuration will be stored in the log file.

each loaded datalog will have its own independant datastream topology (but share a memory layout definition). this increases log loading overhead slightly but not really runtime overhead.

you will be able to choose any combination of messages and vary their polling frequency a bit. having multiple selectable datastreams is a bit tricky for the dashboard and analyzer and all that, it'll just require me to rethink a bit of stuff.

consider when you have a message without RPM in it, and that one is polled often. when there's no RPM data, do you just display the last known RPM? or do you display zero? what if zero is a potentially normal state for that data, that means you're displaying false data. do you just grey the control out and make it blank? what if RPM is only polled every 30 seconds? does it just flash the rpm then turn grey again?... definitely this is the part that requires the most thought, it'll probably take more work than the discovery engine itself.

i'll need a new datalog format, so the next version probably won't load old logs.