Removing windows specific code

iZsh · 2010-02-20 21:56:56

Hello,

I'm almost done with removing all windows specific code. Everything seems to compile and link just fine (I haven't test the QT part yet, but I wouldn't expect much issues with it).

I'm using mingw/msys + pthread-win32 (2.8) + readline 5.x (from gnuwin32) + libusb-win32 and a few changes in my sandbox I'd be willing to commit.

The only showstopper I'm having right now is with libusb, specifically, claiming the device.

After some google-ing, it seems we need to call usb_set_configuration under windows, whereas it's not absolutely mandatory on linux and mac os. When I'm calling usb_set_configuration(handle, 0), the return code is ok, but it still fails to claim the device afterwards. And when I'm calling usb_set_configuration(handle, 1), this function just hangs forever...

Would it be ok if I commit the code (which basically get rid of prox.c and windows specific code and includes, and use the exact same code than mac os and linux... Hence a unique code base for the client side, which is a big step forward maintainability IMHO), with the hope that someone else will help me figuring out this libusb issue? I'm running under VMWare, so I'm not quite sure whether or not it could be the issue.

It compiles and links all the tools just fine (proxmark3, snooper, cli and flasher), only the proxmark device claim is failing. Once this is fixed, hopefully everything else should be just fine since it's the exact same code than mac and linux.

What do you guys think?

marcan · 2010-02-20 22:34:34

I say go for it. Windows USB is a mess, and we can't be maintaining a totally separate codebase for it.

Omikron · 2010-02-20 22:59:18

While the Windows USB code may be a mess, if it's somewhat functional currently I don't see why it would be a good idea to essentially break Windows functionality with no immediate replacement for it.

It seems to me that it would be more desirable to find the appropriate fix and then unify the codebase.

iZsh · 2010-02-20 23:15:05

the issue is that it's very hard to unify it while being the only person doing it.
And your recent issues with flashing under windows just prove it's not really functional, whereas I have absolutely no issue at all under mac os

Anyway, I managed to get pass the hang and claimed the device (I uninstalled the filter version from libusb-win32, and used the "device" version of libusb-win32, wherein you create a custom .inf file for your device using a wizard and then install it as a driver for it).

but now I'm getting this:

read_failed: usb_reap: timeout error (-116)!
Trying to reopen the device...

Investigating again...

Omikron · 2010-02-20 23:21:20

FWIW I don't think my flashing issues have anything to do with how the client is being compiled but rather with how the ARM code is being compiled.

I'm currently flashing via Atmel's official USB JTAG device so it has nothing to do with the USB code in question. In fact as of SVN 351 I was even able to reflash the bootloader and FPGA images fine over USB via the prox client. It's the OS image that I'm having major issues with ATM.

Unfortunately I don't have much programming experience so I can't contribute to the codebase. :-/

Omikron · 2010-02-20 23:31:19

As a side note, here are my current results when trying to compile using SVN 359:

C:\prox-dev\proxmark\cockpit>5makeall.bat
**************
*** armsrc ***
**************
perl ../tools/mkversion.pl .. > version.c || copy ../common/default_version.c version.c
arm-eabi-gcc -c -I../include -I../common -Wall -Werror -pedantic -std=gnu99 -O6 -DWITH_LF -DWITH_ISO15693 -DWITH_ISO1444
3a -DWITH_ISO14443b -I. -mthumb -mthumb-interwork -o obj/version.o version.c
arm-eabi-objcopy -O elf32-littlearm -I binary -B arm --redefine-sym _binary____fpga_fpga_bit_start=_binary_fpga_bit_star
t --redefine-sym _binary____fpga_fpga_bit_end=_binary_fpga_bit_end --prefix-sections=fpga_bit  ../fpga/fpga.bit obj/fpga
.o
arm-eabi-gcc -c -I../include -I../common -Wall -Werror -pedantic -std=gnu99 -O6 -DWITH_LF -DWITH_ISO15693 -DWITH_ISO1444
3a -DWITH_ISO14443b -I. -mthumb -mthumb-interwork -o obj/start.o start.c
arm-eabi-gcc -c -I../include -I../common -Wall -Werror -pedantic -std=gnu99 -O6 -DWITH_LF -DWITH_ISO15693 -DWITH_ISO1444
3a -DWITH_ISO14443b -I. -mthumb -mthumb-interwork -o obj/iso15693.o iso15693.c
arm-eabi-gcc -c -I../include -I../common -Wall -Werror -pedantic -std=gnu99 -O6 -DWITH_LF -DWITH_ISO15693 -DWITH_ISO1444
3a -DWITH_ISO14443b -I. -mthumb -mthumb-interwork -o obj/lfops.o lfops.c
arm-eabi-gcc -c -I../include -I../common -Wall -Werror -pedantic -std=gnu99 -O6 -DWITH_LF -DWITH_ISO15693 -DWITH_ISO1444
3a -DWITH_ISO14443b -I. -mthumb -mthumb-interwork -o obj/hitag2.o hitag2.c
arm-eabi-gcc -c -I../include -I../common -Wall -Werror -pedantic -std=gnu99 -O6 -DWITH_LF -DWITH_ISO15693 -DWITH_ISO1444
3a -DWITH_ISO14443b -I. -mthumb -mthumb-interwork -o obj/appmain.o appmain.c
arm-eabi-gcc -c -I../include -I../common -Wall -Werror -pedantic -std=gnu99 -O6 -DWITH_LF -DWITH_ISO15693 -DWITH_ISO1444
3a -DWITH_ISO14443b -I. -mthumb -mthumb-interwork -o obj/printf.o printf.c
arm-eabi-gcc -c -I../include -I../common -Wall -Werror -pedantic -std=gnu99 -O6 -DWITH_LF -DWITH_ISO15693 -DWITH_ISO1444
3a -DWITH_ISO14443b -I. -mthumb -mthumb-interwork -o obj/util.o util.c
arm-eabi-gcc -c -I../include -I../common -Wall -Werror -pedantic -std=gnu99 -O6 -DWITH_LF -DWITH_ISO15693 -DWITH_ISO1444
3a -DWITH_ISO14443b -I. -mthumb -mthumb-interwork -o obj/usb.o ../common/usb.c
arm-eabi-gcc -c -I../include -I../common -Wall -Werror -pedantic -std=gnu99 -O6 -DWITH_LF -DWITH_ISO15693 -DWITH_ISO1444
3a -DWITH_ISO14443b -I. -mthumb-interwork -o obj/fpgaloader.o fpgaloader.c
arm-eabi-gcc -c -I../include -I../common -Wall -Werror -pedantic -std=gnu99 -O6 -DWITH_LF -DWITH_ISO15693 -DWITH_ISO1444
3a -DWITH_ISO14443b -I. -mthumb-interwork -o obj/legicrf.o legicrf.c
arm-eabi-gcc -c -I../include -I../common -Wall -Werror -pedantic -std=gnu99 -O6 -DWITH_LF -DWITH_ISO15693 -DWITH_ISO1444
3a -DWITH_ISO14443b -I. -mthumb-interwork -o obj/iso14443crc.o ../common/iso14443crc.c
arm-eabi-gcc -c -I../include -I../common -Wall -Werror -pedantic -std=gnu99 -O6 -DWITH_LF -DWITH_ISO15693 -DWITH_ISO1444
3a -DWITH_ISO14443b -I. -mthumb-interwork -o obj/crc16.o ../common/crc16.c
arm-eabi-gcc -c -I../include -I../common -Wall -Werror -pedantic -std=gnu99 -O6 -DWITH_LF -DWITH_ISO15693 -DWITH_ISO1444
3a -DWITH_ISO14443b -I. -mthumb-interwork -o obj/iso14443a.o iso14443a.c
arm-eabi-gcc -c -I../include -I../common -Wall -Werror -pedantic -std=gnu99 -O6 -DWITH_LF -DWITH_ISO15693 -DWITH_ISO1444
3a -DWITH_ISO14443b -I. -mthumb-interwork -o obj/iso14443.o iso14443.c
arm-eabi-gcc -c -I../include -I../common -Wall -Werror -pedantic -std=gnu99 -O6 -DWITH_LF -DWITH_ISO15693 -DWITH_ISO1444
3a -DWITH_ISO14443b -I. -mthumb-interwork -o obj/legic_prng.o ../common/legic_prng.c
arm-eabi-gcc -c -I../include -I../common -Wall -Werror -pedantic -std=gnu99 -O6 -DWITH_LF -DWITH_ISO15693 -DWITH_ISO1444
3a -DWITH_ISO14443b -I. -mthumb-interwork -o obj/crc.o ../common/crc.c
make: *** No rule to make target `../../devkitARM/lib/gcc/arm-elf/4.1.0/interwork/libgcc.a', needed by `obj/fullimage.el
f'.  Stop.
***************
*** bootrom ***
***************
perl ../tools/mkversion.pl .. > version.c || copy ../common/default_version.c version.c
arm-eabi-gcc -c -I../include -I../common -Wall -Werror -pedantic -std=gnu99 -I. -mthumb -mthumb-interwork -o obj/version
.o version.c
arm-eabi-gcc -c -I../include -I../common -Wall -Werror -pedantic -std=gnu99 -I. -mthumb-interwork -o obj/ram-reset.o ram
-reset.s
arm-eabi-gcc -c -I../include -I../common -Wall -Werror -pedantic -std=gnu99 -I. -mthumb-interwork -o obj/flash-reset.o f
lash-reset.s
arm-eabi-gcc -c -I../include -I../common -Wall -Werror -pedantic -std=gnu99 -I. -mthumb-interwork -o obj/fromflash.o fro
mflash.c
arm-eabi-gcc -c -I../include -I../common -Wall -Werror -pedantic -std=gnu99 -I. -mthumb-interwork -o obj/usb.o ../common
/usb.c
arm-eabi-gcc -c -I../include -I../common -Wall -Werror -pedantic -std=gnu99 -I. -mthumb-interwork -o obj/bootrom.o bootr
om.c
arm-eabi-ld -g -Tldscript-flash --oformat elf32-littlearm -Map=obj/bootrom.map -o obj/bootrom.elf obj/version.o obj/ram-
reset.o obj/flash-reset.o obj/fromflash.o obj/usb.o obj/bootrom.o
arm-eabi-objcopy -Osrec --srec-forceS3 --strip-debug --no-change-warnings --change-addresses -0x100000 --change-start 0
--change-section-address .bss+0 --change-section-address .data+0 --change-section-address .commonarea+0 obj/bootrom.elf
obj/bootrom.s19
**************
*** client ***
**************
gcc -std=gnu99 -I. -I../include -I../common -I/opt/local/include -Wall -Wno-unused-function  -g3  -o prox.exe prox.o win
gui.o crc16.o iso14443crc.o data.o graph.o ui.o cmddata.o cmdhf.o cmdhf14a.o cmdhf14b.o cmdhf15.o cmdhflegic.o cmdhw.o c
mdlf.o cmdlfem4x.o cmdlfhid.o cmdlfti.o cmdparser.o cmdmain.o flash.c -lgdi32 -lsetupapi

d18c7db · 2010-02-20 23:55:08

arm-eabi? You must be using devkitARM r27 or later, you need to define LIBGCC=devkitARM-r27\lib\gcc\arm-eabi\4.4.2\libgcc.a (tweak the path as appropriate for your system).

Omikron · 2010-02-20 23:55:49

I'm also curious as to what the expectation is for prerequisites for compiling the Windows client. Am I to use the devkitWIN folder from the current "Windows Compiling Environment" or am I to use MiniGW? Neither of these environments are currently referenced in the 0setpath.bat file and I've had to make manual edits to add the paths.

Omikron · 2010-02-20 23:58:17

d18c7db wrote:

arm-eabi? You must be using devkitARM r27 or later, you need to define LIBGCC=devkitARM-r27\lib\gcc\arm-eabi\4.4.2\libgcc.a (tweak the path as appropriate for your system).

I switched because of notes in the SVN regarding the change:

Revision: 355
Author: marcansoft
Date: 3:28:29 PM, Saturday, February 20, 2010
Message:
Switch compiler to arm-eabi instead of arm-elf

Non-EABI has been deprecated for a while and in fact won't build
with interworking support these days. Configuring the latest
gcc/binutils for arm-eabi works out of the box, so use that.
----
Modified : /trunk/common/Makefile.common

d18c7db · 2010-02-21 00:03:30

You must understand that MinGW is only used to compile the client code in the client directory which makes an executable that runs on a PC.

You need a cross-compiler such as yagarto, devkitARM or whatever you please to compile the proxmark code in bootrom and armsrc which produces executable code that runs on the ARM (the PM3 board). You will always need to use two compilers because you're compiling two sets of executables for two different platforms.

Also generally when replying to posts there's no need to quote the original, especially when it's a large post

iZsh · 2010-02-21 00:08:50

Ok I found the issue, the ETIMEDOUT value was not the same on mingw.

Preparing for my commit then...

Omikron · 2010-02-21 00:15:08

d-

Of course I fully understand what a cross compiler is, what MiniGW is for, and what devkitARM is for. :-)

I was merely stating that for the purposes of someone who wants to take advantage of the batch files to make life easier, there is currently no PATH entry to point to the location of the Windows compiler, in this case MiniGW which by default installs to C:\MiniGW.

It's a pretty minor thing, but I suppose it doesn't merit fixing until the new compiling environment is somewhat stable.

iZsh · 2010-02-21 00:41:53

The new code is committed.
It's a little bit more tedious to setup initially (since you have to install libusb-win32, readline from gnuwin32 and pthread), but at least we have one unique code base now.

It also means the flash procedure is now the same than MacOS and Linux (i.e. using "flasher", prox.exe is gone)

Omikron · 2010-02-21 01:26:01

iZsh, can you provide a basic idea of what packages I need to grab in order to build the environment from scratch? I've muddled something together by grabbing libusb-win32, readline, and pthreads and copying the appropriate include files to where I think they should go but I'm still not able to compile the flasher:

**************
*** client ***
**************
gcc -std=gnu99 -I. -I../include -I../common -I/opt/local/include -Wall -Wno-unused-function  -g3   -c -o proxmark3.o pro
xmark3.c
gcc -std=gnu99 -I. -I../include -I../common -I/opt/local/include -Wall -Wno-unused-function  -g3   -c -o crc16.o ../comm
on/crc16.c
gcc -std=gnu99 -I. -I../include -I../common -I/opt/local/include -Wall -Wno-unused-function  -g3   -c -o iso14443crc.o .
./common/iso14443crc.c
gcc -std=gnu99 -I. -I../include -I../common -I/opt/local/include -Wall -Wno-unused-function  -g3   -c -o data.o data.c
gcc -std=gnu99 -I. -I../include -I../common -I/opt/local/include -Wall -Wno-unused-function  -g3   -c -o graph.o graph.c

gcc -std=gnu99 -I. -I../include -I../common -I/opt/local/include -Wall -Wno-unused-function  -g3   -c -o ui.o ui.c
gcc -std=gnu99 -I. -I../include -I../common -I/opt/local/include -Wall -Wno-unused-function  -g3   -c -o cmddata.o cmdda
ta.c
gcc -std=gnu99 -I. -I../include -I../common -I/opt/local/include -Wall -Wno-unused-function  -g3   -c -o cmdhf.o cmdhf.c

gcc -std=gnu99 -I. -I../include -I../common -I/opt/local/include -Wall -Wno-unused-function  -g3   -c -o cmdhf14a.o cmdh
f14a.c
gcc -std=gnu99 -I. -I../include -I../common -I/opt/local/include -Wall -Wno-unused-function  -g3   -c -o cmdhf14b.o cmdh
f14b.c
gcc -std=gnu99 -I. -I../include -I../common -I/opt/local/include -Wall -Wno-unused-function  -g3   -c -o cmdhf15.o cmdhf
15.c
gcc -std=gnu99 -I. -I../include -I../common -I/opt/local/include -Wall -Wno-unused-function  -g3   -c -o cmdhflegic.o cm
dhflegic.c
gcc -std=gnu99 -I. -I../include -I../common -I/opt/local/include -Wall -Wno-unused-function  -g3   -c -o cmdhw.o cmdhw.c

gcc -std=gnu99 -I. -I../include -I../common -I/opt/local/include -Wall -Wno-unused-function  -g3   -c -o cmdlf.o cmdlf.c

gcc -std=gnu99 -I. -I../include -I../common -I/opt/local/include -Wall -Wno-unused-function  -g3   -c -o cmdlfem4x.o cmd
lfem4x.c
gcc -std=gnu99 -I. -I../include -I../common -I/opt/local/include -Wall -Wno-unused-function  -g3   -c -o cmdlfhid.o cmdl
fhid.c
gcc -std=gnu99 -I. -I../include -I../common -I/opt/local/include -Wall -Wno-unused-function  -g3   -c -o cmdlfti.o cmdlf
ti.c
gcc -std=gnu99 -I. -I../include -I../common -I/opt/local/include -Wall -Wno-unused-function  -g3   -c -o cmdparser.o cmd
parser.c
gcc -std=gnu99 -I. -I../include -I../common -I/opt/local/include -Wall -Wno-unused-function  -g3   -c -o cmdmain.o cmdma
in.c
gcc -std=gnu99 -I. -I../include -I../common -I/opt/local/include -Wall -Wno-unused-function  -g3   -c -o proxusb.o proxu
sb.c
gcc -std=gnu99 -I. -I../include -I../common -I/opt/local/include -Wall -Wno-unused-function  -g3   -c -o guidummy.o guid
ummy.c
gcc   proxmark3.o crc16.o iso14443crc.o data.o graph.o ui.o cmddata.o cmdhf.o cmdhf14a.o cmdhf14b.o cmdhf15.o cmdhflegic
.o cmdhw.o cmdlf.o cmdlfem4x.o cmdlfhid.o cmdlfti.o cmdparser.o cmdmain.o proxusb.o guidummy.o  -L/opt/local/lib -L/usr/
local/lib -lusb -lreadline -lpthread  -o proxmark3
c:\mingw\bin\..\lib\gcc\mingw32\3.4.5\..\..\..\..\mingw32\bin\ld.exe: cannot find -lusb
collect2: ld returned 1 exit status
make: *** [proxmark3] Error 1

iZsh · 2010-02-21 01:51:28

where did you install libusb?
In the libusb-win32-device... tar, you should have include/usb.h and lib/gcc/libusb.a
just copy usb.h to your MinGW's include directory (e.g. C:\MinGW\include) and libusb.a to your MinGW's lib directory (e.g. C:\MinGW\lib).

d18c7db · 2010-02-21 06:43:00

I wonder if it would be a good idea to remove all LCD related code, I added that a long time ago when I hackishly added an LCD to a PM3 board but I have no idea how many, if any, actually have an LCD on their board. Since it's not part of the standard hardware, it may make sense to remove that code.

Omikron · 2010-02-21 09:33:27

d, I would say that removing the LCD would probably not be a terrible idea. I don't know of anyone who has an LCD model either.

iZsh, I've very carefully followed all of the new instructions and set up a completely fresh compiling environment, but I continue to have issues. Everything appears to compile okay (OS Image still seems borked to me) but the client executable files can't see the PM3 board. The board shows up in Device Manager fine but neither flasher.exe or proxmark3.exe seem to be able to talk to the board.

Ideas?

adam@algroup.co.uk · 2010-02-22 02:05:52

Having had similar problems with the new arm-eabi builds under Linux, I've just reverted my compile environment back to gcc-elf (and modified common/Makefile.common to use 'CROSS ?= arm-elf-' instead of 'CROSS ?= arm-eabi-') and everything works fine again...

I guess a bit more detail is required for 'Configuring the latest gcc/binutils for arm-eabi works out of the box, so use that.' before it is safe to try and switch...

Ground Loop · 2010-03-08 02:36:33

Same experience, adam.

The tools/install-gnuarm4.sh script does not provide the arm-eabi-gcc that the tools are looking for:

arm-eabi-gcc -c -I../include -I../common -Wall -Werror -pedantic -std=gnu99 -I. -mthumb -mthumb-interwork -o obj/version.o version.c 
make[1]: arm-eabi-gcc: Command not found

I don't understand why this was checked into the tip code when it obviously doesn't work. I have no "arm-eabi-gcc" anywhere.

The tools are still building for arm-elf, so the hack above is required to build.

adam@algroup.co.uk · 2010-03-08 10:30:44

The tools script pre-dates the switch to arm-eabi... These days you need to install DevkitPRO:

http://wiki.devkitpro.org/index.php/Get … /devkitARM

at some point we need to rationalise all the scripts and docs to take into account the current state of affairs!

iZsh · 2010-03-08 13:19:24

Take a look at COMPILATION.txt. The mac/linux part isn't written yet though.

Proxmark3 community

Announcement

#1 2010-02-20 21:56:56

Removing windows specific code

#2 2010-02-20 22:34:34

Re: Removing windows specific code

#3 2010-02-20 22:59:18

Re: Removing windows specific code

#4 2010-02-20 23:15:05

Re: Removing windows specific code

#5 2010-02-20 23:21:20

Re: Removing windows specific code

#6 2010-02-20 23:31:19

Re: Removing windows specific code

#7 2010-02-20 23:55:08

Re: Removing windows specific code

#8 2010-02-20 23:55:49

Re: Removing windows specific code

#9 2010-02-20 23:58:17

Re: Removing windows specific code

#10 2010-02-21 00:03:30

Re: Removing windows specific code

#11 2010-02-21 00:08:50

Re: Removing windows specific code

#12 2010-02-21 00:15:08

Re: Removing windows specific code

#13 2010-02-21 00:41:53

Re: Removing windows specific code

#14 2010-02-21 01:26:01

Re: Removing windows specific code

#15 2010-02-21 01:51:28

Re: Removing windows specific code

#16 2010-02-21 06:43:00

Re: Removing windows specific code

#17 2010-02-21 09:33:27

Re: Removing windows specific code

#18 2010-02-22 02:05:52

Re: Removing windows specific code

#19 2010-03-08 02:36:33

Re: Removing windows specific code

#20 2010-03-08 10:30:44

Re: Removing windows specific code

#21 2010-03-08 13:19:24

Re: Removing windows specific code

Board footer