new $EE tuning thing!

[lionelhutz]

To me it only makes sense to use an external definition file that can be updated independently of the program code.

[steveo]

of course i see that being useful for a parser designed to deal with more than one ECM, but i'm having trouble finding a real-world use case for it for this particular program.

[steveo]

i am going to use an external file but require it to be in a strict CSV format (no quotes no spaces, no commas allowed in strings). that'll let me maintain the parser very easily and have it parse quickly. i don't expect 99% of users to touch it, ever...

it'll be a single file, and each line will contain some 'meta-data':
- the associated message number
- the device ID (0xf4 or 0xe4)
- a minimum and maximum eehack patch version threshold

internally, each datastream message will be responsible for parsing its own raw CSV data. i wont do a ton of error checking, but enough where you at least get a line number that's at fault if there's a problem.

CSV kinda sucks to write with as it's hard to see columns, so it could be considered an intermediary format. i'll write the definition as an excel spreadsheet, which can easily be exported as a CSV with certain parameters. i'll restrict formulas to a non-optional linear xN+y

i know it's not XML, but it's better than nothing...

[lionelhutz]

That sounds like a simple but good plan. Are you going to throw some kind of text description in there too? That way, someone (or you) can add info and formula to any message without releasing a new version of the program.

[steveo]

if by text description you mean a header, yeah. i'll document it. it'll be 'sorta easy' to understand

here are my current fields

enum {
  DEF_FORMAT_FIELD_DEV_ADDR = 0,
  DEF_FORMAT_FIELD_MSG_NUMBER,
  DEF_FORMAT_FIELD_OFFSET,
  DEF_FORMAT_FIELD_SHORTNAME,
  DEF_FORMAT_FIELD_LONGNAME,
  DEF_FORMAT_FIELD_UOM,
  DEF_FORMAT_FIELD_VIEWLEVEL,
  DEF_FORMAT_FIELD_SIZE,
  DEF_FORMAT_FIELD_PRECISION,
  DEF_FORMAT_FIELD_MULT,
  DEF_FORMAT_FIELD_ADD,
  DEF_FORMAT_FIELD_MINPATCH,
  DEF_FORMAT_FIELD_MAXPATCH,
  DEF_FORMAT_NFIELDS
};

the 'size' and 'precision' define the format. size is in bits and only allows multiples of 8, or 1.

1 bit means its a binary switch, and if so, 'precision' is repurposed as the bit position.

otherwise 8 or higher bits become integers if precision is zero, or floating point if precision is higher than zero.

you're restricted to linear math with MULT and ADD only.

VIEWLEVEL is used for one of three levels, one called HACKER which will include raw or research fields, EXTENDED which is everything a normal person would understand, and BASIC which is just tuning parameters.

there are special view levels, like ERROR which is used to construct lists of error status bits, as they're never displayed in any graphs, tables, or ordinary fields.

i'll include the openoffice version of the spreadsheet with the program, as constructing this without a spreadsheet would be a bitch for sure.

[steveo]

writing this definition file is going to be incredibly painstaking to say the least. im almost done the first message....

[steveo]

im glad im doing this, a complete definition file hasn't even really been properly written yet.

i found that pin D27 is actually a b-body specific EGR position sensor. also found the datastream info for the corvette third o2 sensor. both of these seem to be 'extra appendages' that im going to have to dig into a bit, since neither seem to have associated error codes, does the code even do anything with them?

[troutdog311]

Thanks again for everything Steveo!

[fbody_Brian]

Just wanted to touch base and say that I've been logging and flashing pretty consistently and have no real issues to report. I'm using the latest source from the git repository.
Haven't been very active online much, lot's going on with the car. Pulled the non-functional abs stuff (14lbs), swapped in a cam, installed fiberglass ws6 hood (25lbs).
$EEhack has been invaluable getting the tune straight with the cam.

One issue I do have, I lost my folder of bin files... Had to replace hard drive and my back up is older. I have the bin downloaded from the car, but it's patched, so I have to uncheck insert patches. Any way to unpatch the bin file?

[steveo]

glad it's going well! the next version will be pretty crazy, it's going to read every parameter the ecm can possibly spit out.

Any way to unpatch the bin file?

technically the patches are still 'dumb', you can set the patch byte to zero and it'll rewrite. that check is there to prevent damage in future versions and get people used to how the patching system is 'supposed to work'. the patch version is stored at byte 0x3CB3 on the t-side, just set it to 0x00 and re-flash.

in the future though losing your bins with eehack's patching and flash accelerator enabled is probably a really bad idea......

but also remember when you use my write accelerator it writes some tables to 0xFF. might want to copy your tables to a virgin bin if this bothers you.

[fbody_Brian]

Ok
Thanks
I'll probably just diff a factory bin and copy the changes of known parameters from my currently running bin.

[lionelhutz]

Could you take the patches back out when reading the bin?

[steveo]

potentially can un-patch but I cant un-ff-fill the bin. I want this to be a design policy; keep track of your bins on local storage if you use patches and accelerators.

[Terminal_Crazy]

Hiya Steveo

Just tried logging with the latest EEHack 4.4 earlier.

The Read BLM Cells button in the extended tab doesn't read/displayanything.


Thanks
Mitch

[steveo]

The Read BLM Cells button in the extended tab doesn't read/displayanything.

yeah i meant to disable that. it'll be working in the next version (and moved to a new module with other new 'direct-from-memory' parameters, since it's nothing to do with datalogging). right now it's a bit out of place.

the next version is making progress. i made some blunders that took me forever to debug, but i have the dashboard and the definition API working properly, at least enough to playback/record logs.

i did have to add a field in the definition called 'special'. i hate making meta-parameters but it simplifies things. this can be used to indicate various special field handlers that should be used, such as metric/imperial conversion for speed/temperature, or for a few parameters which require non-linear conversion, this can be defined statically in the program and just called upon with a simple string.

still have -lots- of work to do to turn eehack into an 'all parameters possible' datalogger. it's not just a matter of adding a few tables in some editor and spitting data at them, it's a big fundamental change. here's the short list:

- convert the analyzer and graphing modules to use the new definition api, then finally remove the old api completely
- hack up the event loop so it logs multiple messages effectively in a way that can be rate-controlled (it's half done already)
- make a new parameter selection ui for the graphing module, drop-down boxes by themselves won't do
- get the selective patch version routine working (the defined parameters will only exist in a range of patch versions)
- figure out the UI for the 'extended parameters' tab. both how to select packets to be displayed, and actually displaying them (im thinking tables aren't a bad thing)
- get the error code handler working again
- devise a new logging format that stores multiple messages. im thinking it wont be human readable anymore, but it'll be way more compact. i might use compression too. the next version will not work with old logs, i dont want to support two formats...
- finish the definition file (this is a big one!) anyone willing to help with that? just need excel skills, half a brain, and free time...
- make a loading screen, since loading the def file with hundreds of lines on a slow computer can take a few seconds, it looks bad just sitting there frozen