New flash procedures

henryk · 2009-09-01 16:09:17

Ok, I finally implemented everything I wanted for the flash sequence. New features:

Look ma, no hands! If the new flash tool, the new bootrom and the new osimage are used, a flash sequence can run completely without user interaction, no button pressing required. Pressing the button while the bootrom is active takes precedence over the automatic sequence and releasing the button will abort the bootrom. Don't do that while a flash process is active.
Better protection against accidental misflashes: Both the new flash tool and the new bootrom perform checks on the addresses to write to and abort if something is fishy. The bootrom requires each flash sequence to be preceded with a CMD_START_FLASH (which was defined before but never used) to set up the address area to write to. This protects against trying to flash an os image into the fpga image section for example.
The bootrom protects itself: It will not allow a CMD_START_FLASH that overlaps the bootrom area unless a magic value is set in the command. So should we ever change the partition layout in the future so that a partition overlaps the current bootrom area, this will guarantee that the bootrom is updated first.
The flash tool accepts both S19 files that are relative to 0x0 and those that are relative to 0x100000. The new bootrom will only use 0x100000 to reduce confusion. (The flash tool will translate for old bootroms.) So some time in the future we can drop the ugly translation with objcopy.
The flash tool now autodetects whether the device is already in bootrom mode or still in os mode (even for old firmwares!). This completely replaces the old fast option.
The argument style for the flash tool is now different: first argument is comma-separated list (no spaces!) of firmware areas (bootrom, os, fpga; os and fpga may be abbreviated, bootrom must be given in full), the remaining arguments are file names.

The flip side is that a new bootrom can not be used with an old flash tool, since no protection information is available. (This is a feature, not a bug.) However, I have not yet ported the tool to windows, since I wouldn't be able to test it anyways. Windows users: Do not update your bootrom yet. Somebody will need to look into that.

Once a new windows flash tool is available and some more people got to test the new code, I suggest making a new official release as soon as possible to try and wipe all old bootroms from the face of the earth.

adam@algroup.co.uk · 2009-09-01 18:05:59

Ahahahaha... how ironic! Just when I got around to updating the manual pages with all the nifty 'fast' mode flashing infos...

adam@algroup.co.uk · 2009-09-01 18:18:05

Strange... It's no longer reporting the correct version:

$ ./linux/flasher os,fpga armsrc/obj/osimage.s19 armsrc/obj/fpgaimage.s19
Waiting for Proxmark to appear on USB... Found.
Entering flash-mode...
(You don't have to do anything. Press and release the button only if you want to abort)
Waiting for Proxmark to reappear on USB... Found.
Flashing os from armsrc/obj/osimage.s19
expected = 00120480 flush, c.ext1 = 00120400
done.
Flashing fpga from armsrc/obj/fpgaimage.s19
expected = 0010c4bc flush, c.ext1 = 0010c400
done.
Have a nice day!
addy@nostromo:proxmark3$ ./linux/proxmark3
proxmark3> version
> version
#db# Prox/RFID mark3 RFID instrument
#db# bootrom: svn 201 2009-09-01 17:04:47
#db# os: svn 201 2009-09-01 17:12:30
#db# FPGA image built on 2009/ 8/28 at 23:49:10

surely this should be SVN 206 and dates should match those in my obj directory?

$ ls -l armsrc/obj/
total 828
-rw-r--r-- 1 addy software 149 2009-09-01 18:12 appmain.d
-rw-r--r-- 1 addy software 10372 2009-09-01 18:12 appmain.o
-rwxr-xr-x 1 addy software 50713 2009-09-01 18:12 fpgaimage.elf
-rwxr-xr-x 1 addy software 126582 2009-09-01 18:12 fpgaimage.s19
-rw-r--r-- 1 addy software 158 2009-09-01 18:12 fpgaloader.d
-rw-r--r-- 1 addy software 4924 2009-09-01 18:12 fpgaloader.o
-rw-r--r-- 1 addy software 42623 2009-09-01 18:12 fpga.o
-rwxr-xr-x 1 addy software 145230 2009-09-01 18:12 fullimage.elf
-rw-r--r-- 1 addy software 22080 2009-09-01 18:12 fullimage.map
-rw-r--r-- 1 addy software 155 2009-09-01 18:12 hitag2.d
-rw-r--r-- 1 addy software 11180 2009-09-01 18:12 hitag2.o
-rw-r--r-- 1 addy software 180 2009-09-01 18:12 iso14443a.d
-rw-r--r-- 1 addy software 18084 2009-09-01 18:12 iso14443a.o
-rw-r--r-- 1 addy software 177 2009-09-01 18:12 iso14443.d
-rw-r--r-- 1 addy software 15100 2009-09-01 18:12 iso14443.o
-rw-r--r-- 1 addy software 152 2009-09-01 18:12 iso15693.d
-rw-r--r-- 1 addy software 8728 2009-09-01 18:12 iso15693.o
-rw-r--r-- 1 addy software 170 2009-09-01 18:12 lfops.d
-rw-r--r-- 1 addy software 22128 2009-09-01 18:12 lfops.o
-rwxr-xr-x 1 addy software 112318 2009-09-01 18:12 osimage.elf
-rwxr-xr-x 1 addy software 200126 2009-09-01 18:12 osimage.s19
-rw-r--r-- 1 addy software 143 2009-09-01 18:12 start.d
-rw-r--r-- 1 addy software 872 2009-09-01 18:12 start.o
-rw-r--r-- 1 addy software 136 2009-09-01 18:12 usb.d
-rw-r--r-- 1 addy software 3400 2009-09-01 18:12 usb.o
-rw-r--r-- 1 addy software 140 2009-09-01 18:12 util.d
-rw-r--r-- 1 addy software 3460 2009-09-01 18:12 util.o
-rw-r--r-- 1 addy software 837 2009-09-01 18:12 version.o

(note I have also power cycled the PM3 and re-flashed it twice).

henryk · 2009-09-01 21:07:59

Oh, yes, you need to run svn update first to update the svn version. However, the compilation times should always be correct. Though they are in UTC to reduce confusion.

(SVN has this thing that if you commit, your local repository is not updated automatically. E.g. you won't see your commits in svn log, for examples.)

EDIT: Scratch that first thought. SVN 201 is of course the correct revision number (feature, not bug): It is the last revision in which the firmware was changed, as evidenced by the call to svn info. Revisions 202 through 206 only changed the wiki content, but did not touch the firmware: http://code.google.com/p/proxmark3/source/list

Last edited by henryk (2009-09-01 23:53:12)

XEROEFFECT · 2009-09-02 01:00:28

Thanks Henryk. Your work really is an inspiration. Makes me wish I had as many brain cells as you do.

rleroy · 2009-09-02 01:21:04

I totally agree with XEROEFFECT on this one. The last couple of days, henryk has been on fire with all the major updates to the codebase.

I also gave a glance at the piece of code you modified from the detectreader function (with the light scheme), very nice improvements

d18c7db · 2009-09-02 09:27:21

henryk wrote:

However, I have not yet ported the tool to windows, since I wouldn't be able to test it anyways. Windows users: Do not update your bootrom yet. Somebody will need to look into that.

No worries henryk, I've just commited an updated windows client based on your linux flasher.c code (99% identical code, so was real easy)

Tested OK on my board, see example output below:

P:\Misc\ProxSpaceWork\proxmark\cockpit>..\winsrc\prox bootrom,fpga,os ..\bootrom\obj\bootrom.s19 ..\
armsrc\obj\fpgaimage.s19 ..\armsrc\obj\osimage.s19
Flashing bootrom from ..\bootrom\obj\bootrom.s19
Flashing address: 00101700
done.
Flashing fpga from ..\armsrc\obj\fpgaimage.s19
Flashing address: 0010c400
done.
Flashing os from ..\armsrc\obj\osimage.s19
Flashing address: 0011e700
done.

XEROEFFECT · 2009-09-05 13:06:33

Guys, is there still a temporary breakage for the bootloader as described in the compile page of the wiki? I noticed d18c7db responce above. Does that mean it's good to go for Windows users?

henryk · 2009-09-05 16:06:26

Well, I haven't tried it and just trust that if it works for d18c7db it will work for everyone. That's part of why I was waiting a little longer to see if any problems crop up. Since that doesn't seem to be case I've just now made a release tag: http://proxmark3.googlecode.com/svn/tag … 009_summer with corresponding zip-file 2009.09.05.ProxSpace.zip which includes the windows tools, and pm3-20090905-r216.zip with just the built source code (this directory is included in the ProxSpace.zip and just for convenience).

The upload to the files section on proxmark.org didn't seem to work with no error indication, so I uploaded it into the google code project which seems to be the more proper place anyway. I'll update the documentation now.

XEROEFFECT · 2009-09-05 16:16:37

Henryk, I wish the world was full of people like yourself. Your too good! I really wish I was you. Let's swap chairs.

Last edited by XEROEFFECT (2009-09-05 16:18:07)

rule · 2009-09-05 21:39:06

Thank you Henryk, I've updated the files-section with the new windows-environment and unix/win source-code.

d18c7db · 2009-09-06 14:42:50

With the changes made recently in the way the FPGA image is stored in FLASH, I think I've come across a very obscure lurking bug that may, under some very specific circumstances, come and bite us. This causes a PM3 board to just fail to boot and just sit there like a brick.

I spent most of my free time this weekend setting up Eclipse/yagarto/OpenOCD and got to the point where I can do ICD (In Circuit Debugging) via JTAG which in itself is very cool, when it actually works. Long story short, the problem is in fpgaloader.c at line "DWORD v = FpgaImage[ i ];"

When parsing the FPGA image currently stored in FLASH, everything is fine because the FpgaImage pointer by sheer luck is word aligned, however if it happens not to be, like on my board that uses a different FPGA, that array lookup (essentially a pointer dereference) will cause a DAE (Data Abort Exception) and jump to the data abort vector which is an infinite loop. The board will eventually reset due to the watchdog, boot to the point where it tries to download the FPGA image and cause another DAE, then repeat ad infinitum. See top of page 98 AT91SAM7E datasheet re misaligned addresses.

Currently the fpga.bit starts off as:

00000000h: 00 09 0F F0 0F F0 0F F0 0F F0 00 00 01 61 00 10 ; .............a..
00000010h: 66 70 67 61 2D 70 6C 61 63 65 64 2E 6E 63 64 00 ; fpga-placed.ncd.
00000020h: 62 00 0A 32 73 33 30 76 71 31 30 30 00 63 00 0B ; b..2s30vq100.c..
00000030h: 32 30 30 39 2F 20 38 2F 32 38 00 64 00 09 32 33 ; 2009/ 8/28.d..23
00000040h: 3A 34 39 3A 31 30 00 65 00 00 A4 70 FF FF FF FF ; :49:10.e........

The end of section "e" resolves to a word aligned address 0x4C. If the header was to change slightly such as perhaps the FPGA part number string was to be a single character longer or shorter it would cause the data to shift, for example:

00000000h: 00 09 0F F0 0F F0 0F F0 0F F0 00 00 01 61 00 10 ; ................
00000010h: 66 70 67 61 2D 70 6C 61 63 65 64 2E 6E 63 64 00 ; fpga-placed.ncd.
00000020h: 62 00 0C 33 73 32 35 30 65 76 71 31 30 30 00 63 ; b..3s250evq100.c
00000030h: 00 0B 32 30 30 39 2F 30 39 2F 30 33 00 64 00 09 ; ..2009/09/03.d..
00000040h: 31 31 3A 34 36 3A 34 32 00 65 00 02 95 00 FF FF ; 11:46:42.e......

End of section "e" now resolves to address 0x4e which is no longer word aligned and has caused me major grief trying to find out the cause...

On the plus side, I've learned a bunch of new things so I can't complain too much

henryk · 2009-09-06 20:19:42

d18c7db wrote:

Long story short, the problem is in fpgaloader.c at line "DWORD v = FpgaImage[ i ];"

D'oh, of course! Good work! It's not as if I hadn't had bugs of this type before and should know to avoid them ...
(It did cross my mind momentarily when writing "bitstream_length/4", but then I just thought "Oh well, these images are going to have a multiple of 4 for their size anyway". I completely missed the pointer casting problem.

I've now rewritten it to be byte-based and much cleaner. This shouldn't be a speed problem (I can't distinguish the load times visually), especially since the bitbanging takes much more time than the load from flash. Also this code runs only once, so isn't timing-critical.

Last edited by henryk (2009-09-06 20:20:13)

d18c7db · 2009-09-07 06:59:53

henryk, with the stricter options on the compile now this breaks the windows side compilation:

cc1.exe: warnings being treated as errors
fpgaloader.c: In function 'FpgaGatherVersion':
fpgaloader.c:303: warning: dereferencing type-punned pointer will break strict-aliasing rules
fpgaloader.c:318: warning: dereferencing type-punned pointer will break strict-aliasing rules
fpgaloader.c:322: warning: dereferencing type-punned pointer will break strict-aliasing rules

One way around this which keeps the windows side happy is to turn the void * to char * in the relevant places but I didn't want to make these changes without consulting you in case this might somehow break the linux side.

henryk · 2009-09-08 01:39:18

d18c7db wrote:

One way around this which keeps the windows side happy is to turn the void * to char * in the relevant places but I didn't want to make these changes without consulting you in case this might somehow break the linux side.

Ah, thanks. Yes, using char in these places is better anyways and keeps the casting confined to just one place. Before the next release we should bump the windows toolchain (currently gcc 4.1.0) to be the same as Linux (current 4.3.3) to keep these things from happening.

d18c7db · 2009-09-08 02:29:13

Sounds good, I'm currently using http://www.yagarto.de/ as the toolchain which is based on gcc 4.4.1 which is the defacto Eclipse ARM toolchain it seems. Under that version the compile works up to -O1 but breaks at -O2 and higher so I have to remove the -Werror flag in order to compile with -O6, eg:

P:\Misc\ProxSpaceWork\proxmark\armsrc>arm-elf-gcc --version
arm-elf-gcc (GCC) 4.4.1
Copyright (C) 2009 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

P:\Misc\ProxSpaceWork\proxmark\armsrc>arm-elf-gcc -c -I../include -Wall -Werror -pedantic -std=gnu99 -O2 -mthumb -mthumb-interwork -o obj/appmain.o appmain.c
cc1.exe: warnings being treated as errors
appmain.c: In function 'SendVersion':
appmain.c:252: error: dereferencing type-punned pointer will break strict-aliasing rules

P:\Misc\ProxSpaceWork\proxmark\armsrc>arm-elf-gcc -c -I../include -Wall -Werror -pedantic -std=gnu99 -O1 -mthumb -mthumb-interwork -o obj/appmain.o appmain.c

P:\Misc\ProxSpaceWork\proxmark\armsrc>

I haven't found a tidy way to get around that typecasting/dereferencing you used at that line

henryk · 2009-09-08 03:38:25

Hmm, C aliasing rules seem to be fun: http://gcc.gnu.org/onlinedocs/gcc-4.3.4 … iasing-721
Apparently, using char should bring us into the clear again, so, well ...

Proxmark3 community

Announcement

#1 2009-09-01 16:09:17

New flash procedures

#2 2009-09-01 18:05:59

Re: New flash procedures

#3 2009-09-01 18:18:05

Re: New flash procedures

#4 2009-09-01 21:07:59

Re: New flash procedures

#5 2009-09-02 01:00:28

Re: New flash procedures

#6 2009-09-02 01:21:04

Re: New flash procedures

#7 2009-09-02 09:27:21

Re: New flash procedures

#8 2009-09-05 13:06:33

Re: New flash procedures

#9 2009-09-05 16:06:26

Re: New flash procedures

#10 2009-09-05 16:16:37

Re: New flash procedures

#11 2009-09-05 21:39:06

Re: New flash procedures

#12 2009-09-06 14:42:50

Re: New flash procedures

#13 2009-09-06 20:19:42

Re: New flash procedures

#14 2009-09-07 06:59:53

Re: New flash procedures

#15 2009-09-08 01:39:18

Re: New flash procedures

#16 2009-09-08 02:29:13

Re: New flash procedures

#17 2009-09-08 03:38:25

Re: New flash procedures

Board footer