Write Performance on NOR Flash is getting very low when using the Fault Tolerant Mode with ThreadX, FileX and LevelX

Question

I'm using the NOR-Flash W25Q256 in combination with a STM32H7B3 controller.

After enabling the FaultTolerantMode and writing big data to to the flash, the write performance gets very low.

The W25Q256 has the following coarse specs:

• FlashSize 32MByte
• 8192 physical sectors
• 4kB sector size
• using quadSPI
• used clock frequency: 88MHz

On my controller I'm using threadX, fileX and levelX with the version 6.2.0. Also LX_DIRECT_READ is enabled. and the TX_TIMER_TICKS_PER_SECOND are set to 1000.

To compare different settings I did a write of 8MByte of data to the flash. The write is done by chunks of 512Byte.

for(int i= 0; i< 8192*2; ++i)
{
fx_file_write(&_file, buffer, 512);
}

So, what have I done:

First Step

Measuring write Speed without FaultTolerantMode enabled.

Therefor the settings of the fx_media_format was the following:

fx_media_format(&_media,
                  norDriver,                          // Driver entry
                  nullptr,                               // Device info pointer
                  (UCHAR*)sectorBuffer,       // Media buffer pointer
                  4096,                                  // Media buffer size
                  "C",                                     // Volume Name
                  1,                                        // Number of FATs
                  32,                                      // Directory Entries
                  0,                                        // Hidden sectors
                  (0x2000000 / 0x1000) - 1, // Total sectors
                  0x1000,                              // Sector size
                  1,                                        // Sectors per cluster
                  1,                                        // Heads
                  1);                                       // Sectors per track

With this setting the write speed is about 200kByte/sec. But after 8192 writes of 512Byte fileX returns, that the filesystem is full.

After some investigaton I found out, that levelX expects a logical sector sizes of 512Bytes (LX_NOR_SECTOR_SIZE). This means, that the parameters of fx_media_format seems to be wrong.

Second Step

Measuring write Speed without FaultTolerantMode enabled and fixed parameters of fx_media_format.

The parameters of fx_media_format changed now to:

fx_media_format(&_media,
                  norDriver,                          // Driver entry
                  nullptr,                               // Device info pointer
                  (UCHAR*)sectorBuffer,       // Media buffer pointer
                  4096,                                  // Media buffer size
                  "C",                                     // Volume Name
                  1,                                        // Number of FATs
                  32,                                      // Directory Entries
                  0,                                        // Hidden sectors
                  (0x2000000 / 0x200) - 1, // Total sectors
                  0x200,                              // Sector size
                  8,                                        // Sectors per cluster
                  1,                                        // Heads
                  1);                                       // Sectors per track

With this setting the write speed is about 26.5kByte/sec, which is, compared with the write speed before, very low.

Third Step

Measuring write Speed with FaultTolerantMode enabled and fixed parameters of fx_media_format.

Compared to the second step, I only enabled the fault tolerant mode in the fx_user.h and called the function fx_fault_tolerant_enable(...).

At the beginning of the 8MByte write process the write speed is 23kByte/sec. Compared to the 26.5kByte/sec from the second step it's okay but still very low.

But after some time the NOR-Flash seems to be full and levelX starts to delete obsolete sectors. When the deletion of sectors begin, the write speeds break downs to 8.5kByte/sec.

I investigated some time to check what the problem could be.

I found out the following:

• Every write of 512Bytes leads to 4 writes of 512Bytes in 4 consecutive logical sectors. This explains why a 8Mbyte file will fill more than the whole flash and is why the flash has do some deletion of obsolete sectors.
• The search of the next sector to delete (_lx_nor_flash_next_block_to_erase_find(...) takes a lot of time (30-40msec), because it always iterates through the whole flash before it has found the next sector which can be deleted.

I also played a bit with the possible defines in the fx_user.h and lx_user.h (e.g. LX_NOR_SECTOR_MAPPING_CACHE_SIZE) but nothing had impact on the write speed.

Questions

• Do I have to accept this low writing speed.?
• Is the setting of fx_media_format correct? Because there was the biggest speed loss.
• Is there a possibility to increase the write performance? For my usecase 8.5kBytes/sec are too low.
• What kind of flash (sector size, etc) would be the best for highest write performance?

I already added an issue on the fileX github page (https://github.com/azure-rtos/filex/issues/35)

Thanks for your help.

Patrick

Answer 1

Hello @Patrick,

Just to rule out that there are no issues in the norDriver. Could you confirm it has the sector release logic (refer: case FX_DRIVER_RELEASE_SECTORS:)
[https://github.com/azure-rtos/levelx/blob/master/common/src/fx_nor_flash_simulator_driver.c

In your situation you might want to use a sector size that matches your usage better.  For example, if most of your files were 1KB, then perhaps a 1KB sector size might be good.  The main idea here is that assuming there is one sector per cluster, it might be good performance wise to try to satisfy a majority of the files with a single allocation (one FAT entry - no chain). 

As the flash becomes more utilized, performance will indeed decrease from the initial state. There are a couple of things that you can try to help improve this:

1. Using FileX, we generally suggest that the FileX format leaves one block worth of sectors free, i.e., the format is smaller than the flash size by one flash block.
2. There is a defragment API in LevelX that you can use periodically to spread out the recovery of obsolete blocks. Periodic calls lx_nor_flash_defragment will keep the flash with more free entries such that it doesn't have to reshuffle on every write request.
3. Larger values of LX_NOR_SECTOR_MAPPING_CACHE_SIZE?  By default, this is 16, but it can be changed to a larger power of 2, e.g., 512, 1024.

Please let me know if any of these items help!

Answer 2

Hi @bhnaphade

Thanks for you reply.

Yes, FX_DRIVER_RELEASE_SECTORS is available in the used driver.

To your suggestions:

Using FileX, we generally suggest that the FileX format leaves one block worth of sectors free, i.e., the format is smaller than the flash size by one flash block.

This is already done when I call fx_media_format with the parameter for the total sector amount (0x2000000 / 0x200) - 1, // Total sectors. In this case the -1 is one logical block.

There is a defragment API in LevelX that you can use periodically to spread out the recovery of obsolete blocks. Periodic calls lx_nor_flash_defragment will keep the flash with more free entries such that it doesn't have to reshuffle on every write request.

I have the use case, that a write must be done every second of about 1kByte to a file on the NOR-Flash for 8 - 10 hours. Also during this time other access to other files can be done. So a defragmentation can only be done after this time period. But not during these 8 - 10 hours.

Larger values of LX_NOR_SECTOR_MAPPING_CACHE_SIZE? By default, this is 16, but it can be changed to a larger power of 2, e.g., 512, 1024.

I set LX_NOR_SECTOR_MAPPING_CACHE_SIZE to 1024. But it had no effect. The write speed is still the same. And still gets very low (8.5kByte/sec) when every sector has some content.