Coming Soon: new obd-ii datastream tool

[steveo]

so i just thought i'd take a minute to talk about a project i'm working on in case anyone wants to follow along.

a bunch of people keep harassing me to help with building flash tools for obd-ii based platforms. one problem i have is i don't really own any obd-ii vehicles that i care about. another problem i've witnessed is that all the free obd-ii datalogging tools are total crap. the GM ecms have lots of advanced high speed datalogging features, and barely anything uses those features.

anyway as you probably know i wrote flashhack, which works great for flashing the 'obd 1.5' era ECMs, but to be honest it's really hard to adapt to OBD-ii, because it lives in a world where you ask a question and receive an answer. in the obd-ii world, there could be lots of messages on the bus. also there are lots of different types of interfaces, and lots of different types of ECMs.

i could write a lot of different subclasses in flashhack and maybe make it work, but every new thing would result in a ton of work, it wouldn't be maintainable. i started working on that but i got really BORED.

so i decided to write a new tool that is a bit more generic, kind of like the tunerpro approach.

the tool itself wont do much on its own, but rely on definition files for everything.

imagine for a sec that tunerpro RTs ADX datastream macro thing was good enough to implement something like a flash tool, or a datalogging tool...but also good enough to implement the interface level stuff (ELM, obdxpro, AVT, even homemade tools.) so you wouldn't need a plugin to support something like an ELM interface.

so that's what i'm trying to do right now

i started writing a language that is really basic and fundamental to do all the things that are required, and making a compiler and execution routines for it.
basically a language to manipulate input and output streams, and arrays.

everything in it is centered around 8 bit arrays of data being shuffled between file(s) and a serial interface in various ways, and the design concept of 'send something and wait for possible replies'.

here's the current status of it, everything in the language is currently implemented and working well enough http://fbodytech.com/misc/sdpl-language/

i've implemented the language in a tool that i've proven is good enough to basically 'define' an ELM and obdxpro interface in a simple text file, which means pretty much any interface could be defined in this langauge.

it loads files and can access segments in sequence, which means it does have the fundamentals of a reflashing routine including a kernel and a bin, but there's some more development to do in that area...... and of course a user interface for everything.

more to come soon

[ralmo94]

[steveo]

oh. and it'll work with 8192 baud style stuff too.
.... and i plan to implement custom controls in the scripts
........and also run multiple programs at the same time on the same datastream
so an interested party could do all the stuff that eehack does with simultaneous control and logging..... but for any ecm
hope it works out

[ralmo94]

Baud 8192, does that mean $0D?

[Stokes1114]

0D is 8192.
I'm in for testing for ya Steveo! I've got some 0411 vehicles and various interfaces.

[ras"86]

I wish I could be helpful,sounds like a great project.

[steveo]

right now im a bit stuck on how the user interface is going to be as i'm trying to envision that part before i jump in and start writing it. 'really easy to use' is something that other software already does, so honestly it might be more for advanced users..

[ralmo94]

right now im a bit stuck on how the user interface is going to be as i'm trying to envision that part before i jump in and start writing it. 'really easy to use' is something that other software already does, so honestly it might be more for advanced users..

Would be really cool if it has a dark ui. Bright white screens suck at night.

[steveo]

i've done dark UIs with some of my other software. QT makes it really easy to theme stuff, actually you can write style sheets with it just like a website.

[steveo]

still putting a few hours a day into it but no UI yet

i do have it working so you can build complex structures based on the programs. each program runs in its own thread and has an input and output, so they just need to be chained together. multiple programs operating on the same datastream works too, all the incoming data is replicated, so they can react independently if they were written for it

still need to figure out how a program will generate its controls and stuff like that. i have some good ideas for that but im trying not to make it a 9999 hour project

[steveo]

still working on it, here's a more complex example of the language, this implements obdxpro dvi protocol send/recv from an arbitrary stream:

#--------------------------------------------

#SET $COMMAND AND THIS SUBROUTINE SENDS/GETS REPLY AS $DVI_REPLY - WITH DATA
#ONLY.).  OMIT THE CHECKSUM IN THE $COMMAND INPUT. THE SECOND BYTE(LENGTH) CAN
#BE ANYTHING (00?) AS THIS ROUTINE CORRECTS IT.

*SEND_DVI
SIZE %PKTSIZE $COMMAND
DEC %PKTSIZE
DEC %PKTSIZE
SET $COMMAND[1-1] $PKTSIZE
CHECKSUM_GENERATE %COMMAND "DVI"
TX $COMMAND # .. send
CALL *RECV_DVI
IF FALSE FAIL
ADD %COMMAND[0-0] 10 # reply will have command+10
COMPARE $DVI_REPLY[0-0] | $COMMAND[0-0]
IF = TRUE
IF = RETURN
COMPARE $DVI_REPLY[0-0] | 7F
IF = GOTO *REPLY_ERROR
PRINT "UNKNOWN DVI REPLY: " /HEX $DVI_REPLY
FAIL

*RECV_DVI
RX %DVI_REPLY[2] # GET TWO BYTE HEADER
IF FALSE FALSE
IF FALSE RETURN
SET %_LENGTH_IN $DVI_REPLY[1-1] # SECOND BYTE IS REPLY.
INC %_LENGTH_IN  # +1
RX %_DATA_IN[$_LENGTH_IN]
APPEND %DVI_REPLY $_DATA_IN
CHECKSUM_TEST "DVI" | $DVI_REPLY
IF FALSE FAIL "CHECKSUM ERROR IN REPLY FROM INTERFACE"
PRINT "DVI RX PKT: " /HEX $DVI_REPLY
TRUE
RETURN

*REPLY_ERROR
PRINT "DVI ERROR OCCURRED FOR COMMAND " /HEX $DVI_REPLY[2-2] /TEXT " ERR_CODE " /HEX $DVI_REPLY[3-3]
FAIL "FATAL DVI PROTOCOL ERROR"

[steveo]

so i'm a few hundred hours into it now

making some progress towards additions that will help with rudimentary flash and logging tools.

added support for global lookup tables in the language (which would let you have a file, for example, with names of OBD-II PIDs, and look them up). it compiles to hash tables and is pretty fast. also come in handy for things like interface error codes in logs.

also added support for mapping, skipping, and retrieving regions of an arbitrary array. borrowed some code base from flashhack. that region mapping code was part of the magic of how flashhack could support recovery rom patches as well as efficiently skip regions of FF bytes, etc. without having a huge cluster of code related to doing it. will also make it trivial for a flash program written in this language to actually iterate over the data in a bin.

[steveo]

this may not be exiting to you, but as someone who has spent hundreds and hundreds of hours writing tens of thousand of lines of code to get simple flash routines to work properly, i'm pretty excited about it.

here's how simple it would be to send a bin over the datastream in 32 byte chunks in this language, via the new mapping routine. in this example i will skip 200 bytes at offset 0x0200 (lets pretend they are ram)

the interface (elm, obdxpro, whatever) routines would be running simultaneously, and the program will seamlessly pipe its RX/TX commands to those.

LOAD %DATA "EXAMPLE.BIN"
MAP_CREATE $DATA /MAX_BLOCK_SIZE=32
MAP_SKIP 02 00 /SIZE=200
MAP_REMAINDER # automatically break the rest into <=32byte chunks
*REPEAT  # start of loop
MAP_TAKE %BLK # get next block a BLK
IF FALSE GOTO *FINISHED  # exit loop when finished
TX (write command would go here) $BLK
RX (.. desired reply would go here)
IF TRUE GOTO *REPEAT
*FINISHED

this is a huge portion of a flash routine here.

[ralmo94]

this may not be exiting to you, but as someone who has spent hundreds and hundreds of hours writing tens of thousand of lines of code to get simple flash routines to work properly, i'm pretty excited about it

As someone who has tried to learn coding and spent hours trying to do a simple task, I appreciate all effort you have put forward in all of your freeware.

So am I to understand this will work with a cheap elm327?
Will it also work on vortec black box vcm's?

[steveo]

it could work on anything that someone takes the time to write the interface handler for
but of course it wont make ref lashing with elm better. there are hardware limits on that