HEX file checksum online calculator Firmware for microcontrollers are often stored in Intel HEX files. For testing and debugging sometimes it is necessary to change some values directly in the HEX file. The Intel HEX file format is easy to read and to modify except the checksum. Analyse HEX file line. Pycrc is a CRC reference implementation in Python and a C source code generator for parametrised CRC models. The generated C source code can be optimised for simplicity, speed or small memory footprint, as required on small embedded systems. How To Calculate Hex Checksum; Calculate CRC-8, CRC-16, CRC-32 checksums online for free. (16 Bit) 2: 16 bit Cyclic Redundancy Check. Hex Workshop automatically replaces checksum results generated on a particular file to. Cyclic redundancy check. CRC polynomial: x 32 + x 26 + x 23 + x 22 + x 16 + x 12 + x 11 + x 10 + x 8 + x 7 + x 5 + x 4 + x 2 + x + 1. Used in V.42, MPEG-2. CRC XMODEM: Cyclic redundancy check. CRC polynomial: x 16 + x 12 + x 5 + 1. Used in XMODEM. Custom CRC: Allows you to specify the polynomial values used in checksum calculation.
Loading…
Introduction on CRC calculations
Whenever digital data is stored or interfaced, data corruption might occur. Since the beginning of computer science, people have been thinking of ways to deal with this type of problem. For serial data they came up with the solution to attach a parity bit to each sent byte. This simple detection mechanism works if an odd number of bits in a byte changes, but an even number of false bits in one byte will not be detected by the parity check. To overcome this problem people have searched for mathematical sound mechanisms to detect multiple false bits. The CRC calculation or cyclic redundancy check was the result of this. Nowadays CRC calculations are used in all types of communications. All packets sent over a network connection are checked with a CRC. Also each data block on your hard-disk has a CRC value attached to it. Modern computer world cannot do without these CRC calculation. So let’s see why they are so widely used. The answer is simple, they are powerful, detect many types of errors and are extremely fast to calculate especially when dedicated hardware chips are used.
One might think, that using a checksum can replace proper CRC calculations. It is certainly easier to calculate a checksum, but checksums do not find all errors. Lets take an example string and calculate a one byte checksum. The example string is “Lammert” which converts to the ASCII values [ 76, 97, 109, 109, 101, 114, 116 ]. The one byte checksum of this array can be calculated by adding all values, than dividing it by 256 and keeping the remainder. The resulting checksum is 210. You can use the calculator above to check this result.
In this example we have used a one byte long checksum which gives us 256 different values. Using a two byte checksum will result in 65,536 possible different checksum values and when a four byte value is used there are more than four billion possible values. We might conclude that with a four byte checksum the chance that we accidentally do not detect an error is less than 1 to 4 billion. Seems rather good, but this is only theory. In practice, bits do not change purely random during communications. They often fail in bursts, or due to electrical spikes. Let us assume that in our example array the lowest significant bit of the character ‘L‘ is set, and the lowest significant bit of character ‘a‘ is lost during communication. The receiver will than see the array [ 77, 96, 109, 109, 101, 114, 116 ] representing the string “M`mmert“. The checksum for this new string is still 210, but the result is obviously wrong, only after two bits changed. Even if we had used a four byte long checksum we would not have detected this transmission error. So calculating a checksum may be a simple method for detecting errors, but doesn’t give much more protection than the parity bit, independent of the length of the checksum.
The idea behind a check value calculation is simple. Use a function F(bval,cval) that inputs one data byte and a check value and outputs a recalculated check value. In fact checksum calculations as described above can be defined in this way. Our one byte checksum example could have been calculated with the following function (in C language) that we call repeatedly for each byte in the input string. The initial value for cval is 0.
The idea behind CRC calculation is to look at the data as one large binary number. This number is divided by a certain value and the remainder of the calculation is called the CRC. Dividing in the CRC calculation at first looks to cost a lot of computing power, but it can be performed very quickly if we use a method similar to the one learned at school. We will as an example calculate the remainder for the character ‘m‘—which is 1101101 in binary notation—by dividing it by 19 or 10011. Please note that 19 is an odd number. This is necessary as we will see further on. Please refer to your schoolbooks as the binary calculation method here is not very different from the decimal method you learned when you were young. It might only look a little bit strange. Also notations differ between countries, but the method is similar.
With decimal calculations you can quickly check that 109 divided by 19 gives a quotient of 5 with 14 as the remainder. But what we also see in the scheme is that every bit extra to check only costs one binary comparison and in 50% of the cases one binary subtraction. You can easily increase the number of bits of the test data string—for example to 56 bits if we use our example value “Lammert“—and the result can be calculated with 56 binary comparisons and an average of 28 binary subtractions. This can be implemented in hardware directly with only very few transistors involved. Also software algorithms can be very efficient.
For CRC calculations, no normal subtraction is used, but all calculations are done modulo 2. In that situation you ignore carry bits and in effect the subtraction will be equal to an exclusive or operation. This looks strange, the resulting remainder has a different value, but from an algebraic point of view the functionality is equal. A discussion of this would need university level knowledge of algebraic field theory and I guess most of the readers are not interested in this. Please look at the end of this document for books that discuss this in detail.
Now we have a CRC calculation method which is implementable in both hardware and software and also has a more random feeling than calculating an ordinary checksum. But how will it perform in practice when one ore more bits are wrong? If we choose the divisor—19 in our example—to be an odd number, you don’t need high level mathematics to see that every single bit error will be detected. This is because every single bit error will let the dividend change with a power of 2. If for example bit n changes from 0 to 1, the value of the dividend will increase with 2n. If on the other hand bit n changes from 1 to 0, the value of the dividend will decrease with 2n. Because you can’t divide any power of two by an odd number, the remainder of the CRC calculation will change and the error will not go unnoticed.
The second situation we want to detect is when two single bits change in the data. This requires some mathematics which can be read in Tanenbaum’s book mentioned below. You need to select your divisor very carefully to be sure that independent of the distance between the two wrong bits you will always detect them. It is known, that the commonly used values 0x8005 and 0x1021 of the CRC16 and CRC-CCITT calculations perform very good at this issue. Please note that other values might or might not, and you cannot easily calculate which divisor value is appropriate for detecting two bit errors and which isn’t. Rely on extensive mathematical research on this issue done some decades ago by highly skilled mathematicians and use the values these people obtained.
Furthermore, with our CRC calculation we want to detect all errors where an odd number of bit changes. This can be achieved by using a divisor with an even number of bits set. Using modulo 2 mathematics you can show that all errors with an odd number of bits are detected. As I have said before, in modulo 2 mathematics the subtraction function is replaced by the exclusive or. There are four possible XOR operations.
We see that for all combinations of bit values, the oddness of the expression remains the same. When choosing a divisor with an even number of bits set, the oddness of the remainder is equal to the oddness of the dividend. Therefore, if the oddness of the dividend changes because an odd number of bits changes, the remainder will also change. So all errors which change an odd number of bits will be detected by a CRC calculation which is performed with such a divisor. You might have seen that the commonly used divisor values 0x8005 and 0x1021 actually have an odd number of bits, and not even as stated here. This is because inside the algorithm there is a “hidden” extra bit 216 which makes the actual used divisor value 0x18005 and 0x11021 inside the algorithm.
Last but not least we want to detect all burst errors with our CRC calculation with a maximum length to be detected, and all longer burst errors to be detected with a high probability. A burst error is quite common in communications. It is the type of error that occurs because of lightning, relay switching, etc. where during a small period all bits are set to one. To really understand this you also need to have some knowledge of modulo 2 algebra, so please accept that with a 16 bit divisor you will be able to detect all bursts with a maximum length of 16 bits, and all longer bursts with at least 99.997% certainty.
In a pure mathematical approach, CRC calculation is written down as polynomial calculations. The divisor value is most often not described as a binary number, but a polynomial of certain order. In normal life some polynomials are used more often than others. The three used in the on-line CRC calculation on this page are the 16 bit wide CRC16 and CRC-CCITT and the 32 bits wide CRC32. The latter is probably most used now, because among others it is the CRC generator for all network traffic verification and validation.
For all three types of CRC calculations I have a free software library available. The test program can be used directly to test files or strings. You can also look at the source codes and integrate these CRC routines in your own program. Please be aware of the initialization values of the CRC calculation and possible necessary post-processing like flipping bits. If you don’t do this you might get different results than other CRC implementations. All this pre and post-processing is done in the example program so it should be not to difficult to make your own implementation working. A common used test is to calculate the CRC value for the ASCII string “123456789”. If the outcome of your routine matches the outcome of the test program or the outcome on this website, your implementation is working and compatible with most other implementations.
Just as a reference the polynomial functions for the most common CRC calculations. Please remember that the highest order term of the polynomial (x16 or x32) is not present in the binary number representation, but implied by the algorithm itself.
Polynomial functions for common CRC’s | ||
---|---|---|
CRC-16 | 0x8005 | x16 + x15 + x2 + 1 |
CRC-CCITT | 0x1021 | x16 + x12 + x5 + 1 |
CRC-DNP | 0x3D65 | x16 + x13 + x12 + x11 + x10 + x8 + x6 + x5 + x2 + 1 |
CRC-32 | 0x04C11DB7 | x32 + x26 + x23 + x22 + x16 + x12 + x11 + x10 + x8 + x7 + x5 + x4 + x2 + x1 + 1 |
Literature | ||
2002 | Computer Networks, describing common network systems and the theory and algorithms behind their implementation. | Andrew S. Tanenbaum |
various | The Art of Computer Programming is the main reference for semi-numerical algorithms. Polynomial calculations are described in depth. Some level of mathematics is necessary to fully understand it though. | Donald E. Knuth |
– | DNP 3.0, or distributed network protocol is a communication protocol designed for use between substation computers, RTUs remote terminal units, IEDs intelligent electronic devices and master stations for the electric utility industry. It is now also used in familiar industries like waste water treatment, transportation and the oil and gas industry. | DNP User Group |
One of the things missing for Embedded in the GNU linker is that it cannot generate a CRC checksum. Luckily, there is the solution of using SRecord:
Outline
Because the GNU (e.g. GNU ARM Embedded/launchpad) toolchain does not include a CRC checksum calculation function, I’m showing how the SRecord utility can be used for this. The SRecord tool is an open source project on SourceForge (http://srecord.sourceforge.net/). It is a command line utility which runs on many platforms. I’m using it in this post with Eclipse (e.g. Freescale Kinetis Design Studio, or any other Eclipse based toolchain using the GNU ARM Embedded (Liviu, http://gnuarmeclipse.livius.net/) with GNU for ARM (GNU for ARM Embedded (Launchpad): https://launchpad.net/gcc-arm-embedded). It goes through the steps to create a checksum, add it to the binary image and checking that checksum in the application.
? As the name ‘SRecord’ suggests it deals with S-Records (or S19) files. This is just the default format. SRecord can read and generate pretty much any file format which is used for programming memory devices or microcontroller.
Installation
Go to http://srecord.sourceforge.net/ and download the binaries of your choice from http://srecord.sourceforge.net/download.html. Of course you can as well download the sources and build it yourself. That site hosts as well a lot of good documentation, but if you are a (mostly) Windows user as I am, then be prepared for some Windows user bashing ;-).
SRecord comes with three utilities:
1. srec_info: used to retrieve basic information about the file. It reports things like start address.
2. srec_cmp: used to compare two files. This utility only tells you if two files are (memory-wise) different or not, but not more.
3. srec_cat: This tool is used to extract/add/create/merge/etc files.
All the three tools are command line tools and have extensive support for options. So they easily can be used with make files, scripts or from IDEs as Eclipse. See http://srecord.sourceforge.net/man/index.html for the online manual.
? The cool part is that they support ‘input generators’ and ‘filters’, see http://srecord.sourceforge.net/man/man1/srec_input.html
Generating S-Record Files
Usually the linker main output file is an ELF/Dwarf file which has both code and debug information. The ELF/Dwarf file is used for debugging. All toolchains I’m aware of are able to generate more output files beside of the ELF/Dwarf: S-Record (S19), Intel Hex, etc files. For example in Kinetis Design Studio use the ‘Create flash image’ option in the project settings and press ‘Apply’:
Then you can select the ‘Motorola S-record’ option in the project settings:
The generated S19 file can be found in the Debug output folder:
srec_info
srec_info gives basic information about the file:
gives
See http://srecord.sourceforge.net/man/man1/srec_info.html for further information
srec_cmp
srec_cmp is a program which can compare two files. Unlike a normal diff, it compares two ‘memory’ files. For example
gives
See http://srecord.sourceforge.net/man/man1/srec_cmp.html for further information.
srec_cat
srec_cat is the ‘main’ program of the suite. As the name indicates it can concatenate multiples files. But it can do much more:
-Converting files
-Inserting or removing data
-Joining/splitting files
-Moving data
-Fill in patterns or fill the blanks
-Creating data
-Changing data
-and of course creating multiple kinds of checksums ?
There are many good examples how to use it here: http://srecord.sourceforge.net/man/man1/srec_examples.html
Crop and Generating a Hex Dump
One thing I’m using often is to do a memory dump of my s-record. I can do this with the -hex-dump option:
? After the -Output option there is usually a file name. Using ‘-‘ as file name will write the output to the console.
For example
produdes
I use that approach to quickly inspect or dump content of my image (.elf/.s19) file. Additionally, this allows me to inspect the memory of the target and compare it with what I have in my file.
? The -crop command crops (or cuts) everything out of the data except the range specified (the end address not included).
Filling Memory
Typically a data file as the Motorola S19/SRecord only describes the bytes to be programmed, but not the ‘holes’ or gaps in the memory map. If building a CRC over a memory area with gaps, I need to define it first. For this I can use the -fill command:
To fill multiple areas, e.g. to fill 0x100-0x1FF and 0x300-0x3FF with 0xFF, I could use
Or I could use a list of <start>…<end> addresses: Raw converter 2 5 15.
For the microcontroller on the FRDM-KL25Z board (a KL25Z128) I have the following memory map:
To fill the interrupt table (m_interrupts) and the code (m_text) with 0xFF and dump it, I can use
Generating CRC
srec_cat supports many checksums. The challenge for me was to match the srec_cat way of generating the checksum with the right algorithm and polynom. An online CRC calculation utility on http://www.lammertbies.nl/comm/info/crc-calculation.html helped me to identify the matching CRC polynom and algorithm. In my applications I’m using the Big Endian CCITT CRC16. To generate it, use the following command:
-CRC16_Big_Endian or -crc16-b-e is used to store the CRC is stored in Big Endian format.
CRC16 Source Files
http://www.menie.org/georges/embedded/crc16.html is a great source for a CRC16 calculation function. Another even better one is on http://www.sunshine2k.de/coding/javascript/crc/crc_js.html. I have updated the source to match the 0x1D0F starting point which is used for -CRC16_Big_Endian. I changed the implementation to use a defined starting point: The interface file:
The implementation file:
Checking CRC in the Application
Hex Checksum Calculator
Below is a piece of code in the application which checks the CRC:
The above code assumes that the 16bit CRC is stored at address 0x1FFFE. The application checks the CRC for the code space from address 0x410 up to 0x1FFFD (which does *not* include the CRC itself which is at 0x1FFFE-0x1FFFF).
Using Command Files
Instead passing everything on the command line, I can call the SRecord tool suite programs with the options in an external file. The syntax is
For example
Dumping the CRC for a memory range
Using the ‘@’ syntax, I can execute a script which calculates the CRC and dumps it for me so I can manually check it. For this I use the following content of a command file:
That way I can have things commented (comments start with ‘#’) and keep things readable. The above program calculates the CRC over a given range, stores the value at the artificial address 0x20000 (outside of the code area) and dumps the 16bit value on the console:
Incorporating the CRC Value into the Application
Knowing the CRC, I need to incorporate the CRC value itself into my application. One way would be to do this in the GNU linker script itself (see “FILLing unused Memory with the GNU Linker“). However, that would be a manual process:
1. Determine the CRC value with SRecord and dump it
2. Enter the CRC value into the linker script
Another approach would be:
1. Generate the CRC with SRecord and produce the S19 file which only has the CRC in it (CRC S19 file)
2. Merge the application S19 file with the CRC S19 file
This approach works very well, as srec_cat (as the name indicates) is excellent to concatenate files :-). To produce the CRC S19 file I can use the following command file:
With this, I have only the CRC in FRDM-KL25Z_CRC.srec.
Then I need to fill the unused areas in the application file. Here again, I can generate a ‘filled’ file:
Finally, concatenate the two files with
However, this involves using temporary files which is not ideal. A better approach is to do everything in one step, with a single command file like this:
To me, this is really a cool thing of the SRecord tool: the ability to line up files and content and then merge it ?
And as expected: the application image file has the CRC added:
Adding CRC as Post-Build Step
Now I have the ability to add the CRC to my application file. It would be great if this could be part of my build process? Using normal make files, I simply would call srec_cat after linking. The same thing can be done with Eclipse as ‘Post-Build-Step’. For example to dump the CRC value, I can have this in the project settings:
? Keep in mind that the ‘current directory’ for the build process is the output folder, usually the ‘Debug’ folder.
And it will dump the CRC at the end of the build process:
The same way I can add the CRC to the application:
However, this will fail for a clean build: Grand total 5 2 60.
The reason is that the post build step is executed *before* the S19 file is generated. The solution is to add the generation of the S19 file to the post build step too. It is possible to execute multiple post-build steps (see “Executing Multiple Commands as Post-Build Steps in Eclipse“), just be aware that the separator on Windows is ‘@’ and it is ‘;’ on Linux:
? One strange thing I noticed: as soon as I had multiple commands, I had to use ‘/’ instead of ‘’ on Windows.
With this, everything works as expected:
Summary
Hex File Crc 16 Calculator Form
Generating a CRC is not possible directly with the GNU linker. But this is not a problem, as there is an even more powerful way with the ‘SRecord’ utilities: with the SRecord utilities I can almost any manipulation of the output files I need, plus best of all: it is open source too ?
I have put my project used in this post on GitHub here: https://github.com/ErichStyger/mcuoneclipse/tree/master/Examples/KDS/FRDM-KL25Z/FRDM-KL25Z_CRC
LINKS
-‘SRecord’ home page: http://srecord.sourceforge.net/
-‘SRecord’ examples: http://srecord.sourceforge.net/man/man1/srec_examples.html
-Online CRC calculation: http://www.lammertbies.nl/comm/info/crc-calculation.html
Hex File Converter
-GNU ARM Eclipse Plugins: http://gnuarmeclipse.livius.net/
-GNU for ARM Embedded (Launchpad): https://launchpad.net/gcc-arm-embedded
-Generating S19 Files with CodeWarrior: https://mcuoneclipse.com/2012/09/13/s-record-generation-with-gcc-for-armkinetis/
-Generating S19 Files with Kinetis Design Studio: https://mcuoneclipse.com/2014/04/20/binary-files-for-the-mbed-bootloader-with-eclipse-and-gnu-arm-eclipse-plugins/
Hex File Crc 16 Calculator Download
-CRC16 CCITT Example source code: http://www.menie.org/georges/embedded/crc16.html
-Post-Build-Steps in Eclipse: Executing Multiple Commands as Post-Build Steps in Eclipse
Hex File Checksum Calculator
Retrieved from 'https://gcc-renesas.com/wiki/index.php?title=CRC_Checksum_Generation_with_‘SRecord’_Tools_for_GNU_and_Eclipse&oldid=53'