i can't believe nobody realized the aldl heartbeat function in eehack is totally broken for any bus master other that the CCM, it assumed the checksum was valid for a corvette config only. fbodies have been connecting very slowly as a result...
new program can now auto-connect for any aldl bus master. will have to merge that into eehack later.
i had a closer look at your patch and i think you did some really good work here. i wrote it quickly to a bin and programmed it, and it indeed does seem to respond to a mode 5 request. i haven't written it into a flash routine yet or tested it further. the advantage of this is incredible, about 4% of the bin has to be complete before we're in the 'safe zone' for anything (computer crash, ecm crash, battery dies, whatever). if this works as well as i think it will, our window of failure is very small.Originally Posted by kur4o
Great it worked out. I have left some code for cutting so here is the stage2 test.
4269-4314
b9fa-bd9f
eeb5-eeba
ff86-ff91
fffe-ffff
Patch stays the same.
Great you fixed that. Adding some improved logic for handling idle bus messages will be great for future use where there will be a ton of them. Now it is not perfect and decoding the start- end of the messages is not always optimal.
the other thing... the windows FTDI driver has 16ms of latency by default and that's longer than a heartbeat gap, so unless you lower it to 1ms, the heartbeat function will only work sometimes. im looking for a way to lower that latency in software now.
i think i can solve this in software:
my software can access the vid/pid/serial to determine the current interface, look in the registry to see what the default value is, and if it sucks, i can change it, then change it back after......... there seems to be no other api option for modifying it.The latency timer is a form of time-out mechanism for the read buffer of FTDI devices. When a FT_Read instruction is sent to the device, data will not be sent back to the host PC until the requested number of bytes has been read. If the requested number of bytes never comes, the device would not send data back.
The latency timer counts from the last time data was sent back to the PC. If the latency timer expires, the device will send what data it has available to the PC regardless of how many bytes it is waiting on. The latency timer will then reset and begin counting again.
The default value for the latency timer is 16ms. This value may be customised by adding or changing the following entries in the FTDIPORT.INF file of the driver before installation.
[FtdiPort232.NT.HW.AddReg]
HKR,,"LatencyTimer",0x00010001,50
This example will set the default latency timer value to 50ms. The valid range for the latency timer is 1ms - 255ms, although 1ms is not recommended as this is the same as the USB frame length.
The latency timer value is held in the registry under
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Enum\F TDIBUS\{Device VID, PID and serial number}\0000\Device Parameters\LatencyTimer
making more progress with reliable aldl bus code. i don't think i really spent the time to understand how all this works before.
first we need knowledge of the response time of our serial interface to figure out the best approach for gaining bus master status. anything with latency greater than a few milliseconds can't time the heartbeat properly but WILL work (just harder to connect as we can read the heartbeat, but by the time we get the data, it's possible we're already too far past it). i will find a way to measure the latency but for FTDI interfaces we can read it from the registry.
the new routine i'm using will always listen for idle traffic first, if some exists, it'll determine the bus master and assume control. it'll always recheck the receive buffer between commands, so if something re-assumes bus master status, it'll automatically re-gain it.
the new routine should work equally well with any gm 8192 baud aldl bus that uses master/slave topology with idle traffic, and auto-detect the current bus master. the new logic will also inherently handle cases where there is no current bus master (important for flash recovery)
I'm figuring we will now have any of the following states to handle for initializing the flash routine... i will develop a function to probe for each of them.
- Factory state (full bin booted up and everything is okay)
- Software recovery (kernel is loaded and eeprom in unknown state, but we can probably start over. in this case i want to 'resume' rather than 'restart' the flash routine.. i think i have ideas how to do that.)
- Hardware recovery (kur4o's awesome new mini-loop is booted up, and eeprom is in an unknown state.)
- Bricked (ECM is powered on but unresponsive.... it died somewhere between 'erase' and 'recovery routine installed', so we basically have to apologize to the user at this point)
Absolutely amazing stuff so far. I've been following closely and this stuff is awesome.
So if I'm reading you correctly, does that mean you can inherently support non-LT1 or pre-$EE ALDL communications? Say, for datalogging a '92 base Corvette or ZR-1? That'd be pretty impressive if so.
Even if not, just having more reliable comms and recoverable flashing for $EE is exciting enough. I don't own any non-$EE cars (yet), so I'm super excited! Thank you and kur4o for everything you do!
1990 Corvette (Manual)
1994 Corvette (Automatic)
1995 Corvette (Manual)
i'm not 100% sure that all 8192 baud cars use the same protocol for the bus master 'heartbeat', can anyone confirm?
EE uses: 0xF0 [LENGTH] [BUS_MASTER_ID] [CHECKSUM]
if they do, then yeah, it should work with any of them equally well.
making some good progress on the functional part of the reflash routine.
the code is much more simple and readable due to the better (and more generalized) communication and messaging API, so the flash routine doesn't really have to deal with much ALDL stuff. eehack's flash routine is written with the worst possible coding practices as it's implemented in a tool that was only ever intended to do basic datalogging.
the most difficult part of loading any chunk of executable code is that the aldl protocol limits us to 173 bytes, minus 6 bytes of overhead for a mode 0x06 request.. so you need to write a large routine in multiple chunks, but the only option that mode 6 gives us is 'upload then execute'. so to upload without executing, we need to prepend a chunk of executable code before our payload that saves the stack pointer so we can return to normal operation after the upload is complete, and then apply a negative offset by the size of the header so our upload ends up in the right spot. if you handle the offsets correctly, you can upload a large program backwards, each new chunk will overwrite the 'header' of the last one, and then the last chunk of code you upload, you don't write a header that jumps off, and it'll run the whole thing.
so one of the biggest additions first off will be dynamic loading and execution of large chunks of executable code to the ecm.. we'll be able to upload any size of executable code to ram and it'll handle the headers and offsets functionally. in eehack this is done statically rather than functionally.
i know it doesn't sound useful yet but wait until we next modify the kernel...it'll make it almost effortless.
Last edited by steveo; 03-27-2020 at 11:40 PM.
Loading flash routines on the fly instead of hard coding them is much better in the long term. It opens the door of much bigger and improved flash routines. Adding some parsers to load an external files of flash routines will make it a breeze to test and improve.
With 192 bytes message buffers we have tons of free space for code upload. The usable range is 0-3ff and 1810-1fff.
i thought of that, i have a generic 'bin file' management class, and the request/reply system for communication is capable of mapping and offsetting of sections of those bins to sections of the request's payload. loading any executable payload via a mode 6 request should be a two liner.
bin_file bin(location_of_file)
if ( load_executable_code(device, bin, start, end, offset) == false ) then { stop what we're doing .... }
it should be smart enough to know if it'll fit into a mode 6 message or if it requires multiples, figure the offsets, generate its own header, break it up from there.
Forgive me if this is an off-topic ask, but you mentioned connection quality and I got to thinking, eehack has a really handy feature where you start with the current binfile to look for deltas and see just how much actually changed in the new file; could you test that connection quality with the binfile read and then tell the user the connection quality before any writes are even made, or is that not possible? Or is there an unused spot in memory like RAM that could be written and read to 100 times before an actual erase/write to determine connection quality?
The problem is that reading back EEPROM data requires uploading a small program which executes the read routine. While there's nothing stopping you from uploading a small program to do a smaller version of it--and in fact that's partly what's being discussed above--the question is how useful that will actually be. For now you should know that the ALDL protocol, like just about any decent serial communication protocol, includes a checksum at the end of every message as well as a byte that tells you the expected length of said message. If these two values do not match the data actually received, then voila, you know that the connection is bad. You may not know why, but you know that you aren't getting what you expect.
So for now, I think it's far more important to focus on the reconstruction of the basic ALDL communication routine and the development of a nearly brick-proof flash routine than it is engineering a new stability test routine. But that's just me.
1990 Corvette (Manual)
1994 Corvette (Automatic)
1995 Corvette (Manual)
testing connection quality is definitely something that i think is a good idea before starting a flash routine. eehack has that stability test that just performs (part of) a flash read, under the premise that if it fails, probably a write would fail too, most of the uploaded code is the same. i'll always put a similar option in.
Posting Permissions
Reply With Quote
Bookmarks