- \r\n
- Write acceleration for Arduino boards<\/a><\/li>\r\n<\/ul>\r\n<\/li>\r\n
- SD cards as data logger<\/a><\/li>\r\n
- Using multiple SD card modules<\/a><\/li>\r\n<\/ul>\r\n\n
Why do you need SD cards?<\/h2>\n\n
There are many ways to store data using microcontrollers:<\/p>\r\n
- \r\n
- Smaller amounts of data ranging from a few hundred bytes to a few kilobytes can be stored on the internal EEPROM. That is what I have described here<\/a>.<\/li>\r\n
- For somewhat larger amounts of data up to ~1 megabyte you can use external EEPROMs. These are characterized by low space and power requirements. I described external, I2C-controlled EEPROMs here<\/a>. An article about the faster, SPI-based representatives can be found here<\/a>.<\/li>\r\n
- If you use an ESP32 or ESP8266 based board, you can store several megabytes of data using LittleFS (formerly SPIFFS = SPI Flash File Storage).<\/li>\r\n
- Finally, there are various options for storing data in the cloud or sending it via a network to a PC, for example.<\/li>\r\n<\/ul>\r\n
Why do you need SD cards then?<\/strong> There can be several reasons:<\/p>\r\n
- \r\n
- The SD cards are characterized by high storage capacities in the range of gigabytes.<\/li>\r\n
- The data is conveniently transportable. This allows you to read and evaluate the stored data with a card reader on your PC, while your devices (MCU, sensors, etc.) remain on site.<\/li>\r\n
- Unlike online storage methods, you are not dependent on network availability.<\/li>\r\n<\/ol>\r\n\n
SD card formats, classes and modules<\/h2>\n\n
SD cards (“S<\/strong>ecure D<\/strong>igital memory cards”) differ among others in:<\/p>\r\n
- \r\n
- their dimensions,<\/li>\r\n
- their maximum storage capacity and<\/li>\r\n
- their minimum write speed.<\/li>\r\n<\/ul>\r\n
Strictly speaking, the term “SD card” means the large models with dimensions of 24 mm x 32 mm x 2.1 mm and a maximum storage capacity of 2 gigabytes. The somewhat clunky SD card has been replaced by the small microSD card in the 11 mm \u00d7 15 mm \u00d7 1.0 mm format in most applications. Regarding the size, there are still the rarely used miniSD cards in between.<\/p>\r\n
Regarding the maximum storage capacities, a distinction is made between SD, SDHC, SDXC and SDUC cards. Especially for the SD and the microSD\/microSDHC cards there are cheap modules for Arduino and other boards, for example these: <\/p>\r\n
<\/p>\r\n\n
![\"Left:](\"https:\/\/wolles-elektronikkiste.de\/wp-content\/uploads\/2022\/11\/sd_holder_and_sd_cards-1024x378.jpg\")
<\/a>Left: microSD\/microSDHC module up to max. 32 GB, right: SD card module up to 2 GB. <\/figcaption><\/figure>\n With a maximum of 32 gigabytes, SDHC cards have a much higher storage capacity than SD cards.<\/p>\r\n
The minimum writing speed can be derived from the class of the SD card. This is the small number (2 to 10) in the unclosed circle. The number specifies the speed in megabytes per second.<\/p>\r\n\n
Power consumption<\/h4>\n\n
I determined a power consumption of 1.8 milliamps in idle mode for the microSDHC card module. For battery-powered projects, this is quite high. You could consider switching the module on and off as needed via a transistor, or better via a MOSFET<\/a>. During writing, the power consumption is just under 40 milliamperes.<\/p>\r\n\n
microSD Shield for the Wemos D1 mini<\/h4>\n\n
A particularly practical solution is available for the Wemos D1 Mini Board, namely a microSD card shield:<\/p>\r\n
\n![\"Wemos](\"https:\/\/wolles-elektronikkiste.de\/wp-content\/uploads\/2022\/11\/wemos_d1_mini_sd_card_shield-1024x442.jpg\")
<\/a>Wemos D1 Mini microSD Shield<\/figcaption><\/figure><\/div>\n Preparations<\/h2>\n\n
Formatting SD cards<\/h3>\n\n
If you buy an SD card, it will be pre-formatted. The format can<\/em> harmonize with the memory card modules, but it does not have to<\/em>. But even if you want to recycle an old SD card, e.g. from a camera, this is not possible without further ado, since manufacturer-specific formats are sometimes used there.<\/p>\r\n
For the use in the SD card modules, you need the FAT16 or FAT32 format. You can do the formatting on your PC or laptop. If you don’t have an SD card slot, then you should get a USB SD card reader for about 10 Euros. <\/p>\r\n
As software, I recommend the free program SD Card Formatter, which you can get here<\/a>. You have to “unzip” the installation file, and then you can install the program.<\/p>\r\n
SD Card Formatter, with its few settings, is about the simplest program I have ever used. The SD card is automatically detected by the program. Select the option “Overwrite format” and click on “Format”. The process takes a few minutes. And that’s it.<\/p>\r\n
\n![\"Formatting](\"https:\/\/wolles-elektronikkiste.de\/wp-content\/uploads\/2019\/04\/SD_Card_Formatter.jpg\")
<\/a>SD Card Formatter program interface<\/figcaption><\/figure><\/div>\n The SD library<\/h3>\n\n
The SD library is part of the basic Arduino IDE and therefore does not need to be installed via the library manager. Strictly speaking, there is not “the one” SD library, but different versions for different boards. Especially the version for the ESP32 has a few peculiarities, which I will discuss later.<\/p>\r\n
Depending on the board you have set, you will find different example sketches. Unfortunately, the “Examples for any Board” are not hidden when using ESP32 based boards, although they do not work with them. Go to the ESP32 examples \u2192 SD(esp32) instead. <\/p>\r\n\n
![\"Selection](\"https:\/\/wolles-elektronikkiste.de\/wp-content\/uploads\/2022\/11\/arduino_sd_examples-1024x417.png\")
<\/a>Selection of example sketches for different boards<\/figcaption><\/figure>\n Connecting the microcontroller<\/h3>\n\n
The permissible supply voltage for SD cards is usually 2.7 – 3.6 volts. Some SD cards can also tolerate 1.8 volts. Most SD card modules have voltage regulators and can therefore be operated with 5 volts. The microSD card module I use (absurdly) does not have a 3 volt pin, so it must<\/em> be powered by 5 volts.<\/p>\r\n
The communication with the microcontroller takes place via SPI. So, MISO is connected to MISO, MOSI to MOSI, SCK to SCK and CS to CS. You can set the CS pin yourself on most boards.<\/p>\r\n
For the Arduino Nano the wiring looks like this:<\/p>\r\n\n
![\"A](\"https:\/\/wolles-elektronikkiste.de\/wp-content\/uploads\/2022\/11\/SD_Card_Wiring-1024x347.png\")
<\/a>A microSD card module on the Arduino Nano<\/figcaption><\/figure>\n Check your setup<\/h3>\n\n
If you are using an Arduino board, then you can check the circuit and the SD card with the CardInfo sketch from the examples. Do not forget to adjust the CS pin if necessary.<\/p>\r\n
The sketch tells you the SD card type, format, storage capacity and the directories and files present:<\/p>\r\n\n
![\"Output](\"https:\/\/wolles-elektronikkiste.de\/wp-content\/uploads\/2022\/11\/output_cardinfo.png\")
<\/a>Output of CardInfo.ino<\/figcaption><\/figure>\n CardInfo does not work on ESP32 or ESP8266 boards. As a first test for an ESP32 based board, I recommend instead the example sketch SD_Test.ino and for an ESP8266 board the sketch listfiles.ino.<\/p>\r\n\n
Writing and reading<\/h2>\n\n
Now let’s have a look at a few functions of the SD library and start with simple read and write operations.<\/p>\r\n\n
Writing<\/h3>\n\n
The following example sketch writes an integer array to the SD card. Again, (but for the last time in this post) the hint: You may have to adjust the CS pin. <\/p>\r\n\n
#include <SD.h>\r\nconst int chipSelect = 10; \/\/ choose an appropriate pin for your board\r\n\r\nvoid setup(){\r\n String myFile = \"integers.txt\"; \/\/ for ESP32: \"\/integers.txt\";\r\n int intArray[10] = {42, 424, 4242, 17, 1234, 56, 1967, 299, 3333, 5678};\r\n Serial.begin(9600);\r\n\/\/ while(!Serial){} \/\/ needed for some boards\r\n\/\/ delay(1000); \/\/ helps in case of incomplete output on serial monitor\r\n \r\n if(!SD.begin(chipSelect)){\r\n Serial.println(\"SD-Card not connected!\");\r\n while(1);\r\n }\r\n else{\r\n Serial.println(\"SD-Card initialized\");\r\n }\r\n \r\n SD.remove(myFile);\r\n File dataFile = SD.open(myFile, FILE_WRITE);\r\n\r\n if(dataFile){\r\n for(int i = 0; i < 10; i++){\r\n dataFile.println(intArray[i]);\r\n }\r\n dataFile.close();\r\n Serial.println(\"Data written\");\r\n }\r\n else{\r\n Serial.println(\"Could not open file\");\r\n }\r\n}\r\n\r\nvoid loop(){}<\/pre>\r\n\nExplanations for sd_card_write.ino <\/h4>\n\n
The SD library is included with
#include <SD.h><\/code>. The initialization is done withSD.begin(chipSelectPin)<\/code>. If this process works, thenSD.begin()<\/code> returns the valuetrue<\/code>, otherwisefalse<\/code>. You do not need to create an SD object unless you want to use two or more SD cards (see last chapter).<\/p>\r\nThe file we create and fill is called “integers.txt”. We store the name in the string variable “myFile”. Only 8 characters are allowed for the file name, plus 3 characters for the file extension – welcome to the last millennium!<\/p>\r\n
The instruction
SD.remove(myFile);<\/code> deletes the file myFile (or “integers.txt”) if it should exist. Without this measure, the data to be saved would be appended to the end of the file again with each program run (except with the ESP32). <\/p>\r\nLine 20 appears inconspicuous, but there is much behind it:<\/p>\r\n\n
File dataFile = SD.open(myFile, FILE_WRITE);<\/pre>\r\n\n
For understanding (or confusion?):<\/p>\r\n
- \r\n
- SD is an object of SDClass. This is not obvious since you did not have to create SD yourself.<\/li>\r\n
- File is a property of SDClass. <\/li>\r\n
SD.open(myFile)<\/code> opens the path to “myFile”. The function returns an object of the File class, i.e. a file object, which is given the name dataFile. The FILE_WRITE parameter specifies that the file is opened for writing.<\/li>\r\n- So “myFile” contains only the filename or the path to the file. In contrast, “dataFile” is the actual file object.\r\n
- \r\n
- Consequently,
dataFile.name()<\/code> returns the content of the variable “myFile”, in this case “integers.txt”. You can try that out.<\/li>\r\n<\/ul>\r\n<\/li>\r\n<\/ul>\r\nThe instructions for writing are
print()<\/code>,println()<\/code> or alternativelywrite()<\/code>. The similarities to Serial functions are not accidental because File and Serial are relatives, namely “children” of the Stream class.<\/p>\r\nFinally, you complete the writing process with
close()<\/code>.<\/p>\r\n\nPeculiarities of the ESP32 <\/h4>\n\n
For the ESP32 you need to consider:<\/p>\r\n
- \r\n
- The names of files and directories must be preceded by a slash (“\/”).<\/li>\r\n
- If “FILE_WRITE” is applied to an existing file, then its contents will be overwritten. You can append content using the FILE_APPEND parameter. The SD library for Arduino boards does not know the parameter “FILE_APPEND” and appends the content automatically.<\/li>\r\n
SD.begin()<\/code> can be called without CS Pin. In this case the default GPIO5 is used.<\/li>\r\n<\/ul>\r\n\nRead<\/h3>\n\n
The following sketch reads the just saved data from the SD card:<\/p>\r\n\n
#include <SD.h>\r\nconst int chipSelect = 10; \/\/ choose an appropriate pin for your board\r\n\r\nvoid setup(){\r\n char myFile[] = \"integers.txt\"; \/\/ for ESP32: \"\/integers.txt\";\r\n Serial.begin(9600);\r\n\/\/ while(!Serial){} \/\/ needed for some Boards\r\n\/\/ delay(1000); \/\/ helps in case of incomplete output on serial monitor\r\n \r\n if(!SD.begin(chipSelect)){\r\n Serial.println(\"SD-Card not connected!\");\r\n while(1);\r\n }\r\n else{\r\n Serial.println(\"SD-Card initialized\");\r\n }\r\n\r\n File dataFile = SD.open(myFile);\r\n\r\n if(dataFile){\r\n while(dataFile.available()){\r\n String line = dataFile.readStringUntil('\\n');\r\n Serial.println(line);\r\n }\r\n\/\/ Alternative: read as Integer\r\n\/\/ while(dataFile.available()){\r\n\/\/ int readInt = dataFile.parseInt();\r\n\/\/ Serial.println(readInt);\r\n\/\/ }\r\n dataFile.close();\r\n }\r\n else{\r\n Serial.println(\"Could not open file\");\r\n }\r\n}\r\n\r\nvoid loop(){}\r\n<\/pre>\r\n\nExplanations for sd_card_read.ino<\/h4>\n\n
Much of this is similar to the writing sketch discussed earlier. I will only go into the main differences.<\/p>\r\n
- \r\n
- This time I defined “myFile” as a character array. The reason is merely to show that it can be done.<\/li>\r\n
- By calling
SD.open(myFile)<\/code> without “FILE_WRITE”, the file can only be read.<\/li>\r\n- There are two options for reading the data:\r\n
- \r\n
readStringUntil('\\n')<\/code> reads all characters up to the next line break as a string.<\/li>\r\n- To read out the data as an integer, the practical function
parseInt()<\/code> can be used. For floating-point numbers,parseFloat()<\/code> would be available.<\/li>\r\n<\/ul>\r\n<\/li>\r\n<\/ul>\r\nFinally, the file is closed again with
close()<\/code>.<\/p>\r\n\nThe output should not surprise:<\/p>\r\n\n
![\"Output](\"https:\/\/wolles-elektronikkiste.de\/wp-content\/uploads\/2022\/11\/output_sd_card_read.png\")
<\/a>Output of sd_card_read.ino<\/figcaption><\/figure>\n Creating \/ deleting \/ displaying files and directories <\/h2>\n\n
You have just seen how to create and delete files. Now let’s look at how you do the same with directories and how you display entire directory structures, including the files preserved in them. First here is the sketch which also works on an ESP8266 but not on an ESP32:<\/p>\r\n\n
\r\n\/* Based on:\r\n listfiles.ino, created Nov 2010 by David A. Mellis;\r\n modified 9 Apr 2012 by Tom Igoe\r\n modified 2 Feb 2014 by Scott Fitzgerald\r\n*\/\r\n#include <SD.h>\r\nconst int chipSelectPin = 10;\r\n\r\nvoid setup() {\r\n Serial.begin(9600);\r\n while (!Serial) {}\r\n \r\n Serial.print(\"Initializing SD card...\");\r\n\r\n if (!SD.begin(chipSelectPin)) {\r\n Serial.println(\"initialization failed!\");\r\n while (1);\r\n }\r\n Serial.println(\"initialization done.\");\r\n\r\n Serial.println(\"Adding files:\");\r\n SD.mkdir(\"New_Fold\"); \/\/ make new folder\r\n File dataFile = SD.open(\"file_1.txt\", FILE_WRITE);\r\n dataFile.println(\"Some Content\");\r\n dataFile.close();\r\n\r\n dataFile = SD.open(\"new_fold\/file_2.txt\", FILE_WRITE);\r\n dataFile.println(\"Some other Content\");\r\n dataFile.close();\r\n\r\n File root = SD.open(\"\/\");\r\n printDirectory(root, 0);\r\n\r\n Serial.println();\r\n Serial.println(\"After removing files:\");\r\n SD.remove(\"file_1.txt\");\r\n SD.remove(\"new_fold\/file_2.txt\");\r\n SD.rmdir(\"new_fold\");\r\n\r\n root = SD.open(\"\/\");\r\n printDirectory(root, 0);\r\n\r\n Serial.println(\"done!\");\r\n}\r\n\r\nvoid loop() {}\r\n\r\nvoid printDirectory(File &dir, int numTabs) {\r\n while (true) {\r\n\r\n File entry = dir.openNextFile();\r\n if (! entry) {\r\n \/\/ no more files\r\n break;\r\n }\r\n for (uint8_t i = 0; i < numTabs; i++) {\r\n Serial.print('\\t');\r\n }\r\n Serial.print(entry.name());\r\n if (entry.isDirectory()) {\r\n Serial.println(\"\/\");\r\n printDirectory(entry, numTabs + 1);\r\n } else {\r\n \/\/ files have sizes, directories do not\r\n Serial.print(\"\\t\\t\");\r\n Serial.println(entry.size(), DEC);\r\n }\r\n entry.close();\r\n }\r\n}<\/pre>\r\n<\/p>\r\n<\/div>\r\n\n
Explanations for add_remove_list_files_and_folders.ino<\/h3>\n\n
The setup()<\/h4>\n\n
- \r\n
SD.mkdir(\"New_Fold\")<\/code> creates a directory with the name “New_Fold”.<\/li>\r\n- We create the file “file_1.txt” only to have something more to display.<\/li>\r\n
SD.open(\"New_Fold\/file_2.txt\", FILE_WRITE);<\/code> creates the file “file_2.txt” in the folder “New_Fold”.<\/li>\r\n- The assignment
File root = SD.open(\"\/\");<\/code> is perhaps a little surprising, since a file object called “root” is created, which represents the main directory. Users of Linux or Unix should be less surprised because for these systems a directory is just a special file. <\/li>\r\n- The files and directories are displayed using the
printDirectory()<\/code> function. We’ll get to that in a moment.<\/li>\r\n- We remove the added files and directories with
SD.remove()<\/code> andSD.rmdir()<\/code> respectively.<\/li>\r\n- After that, the content of the SD card is output once again.<\/li>\r\n<\/ul>\r\n\n
The printDirectory() function<\/h4>\n\n
The function
printDirectory()<\/code> is passed a file object (on the first call this is root) and the number of tabs. The number of tabs is used for formatting and represents the current directory depth.<\/p>\r\nFile entry = dir.openNextFile();<\/code> opens the next file found on the SD card and creates the file object “entry” from it. Three scenarios are possible:<\/p>\r\n- \r\n
- There is no other file or directory (
!entry<\/code>). The loop is canceled.<\/li>\r\n- Another entry is available and it is a directory (
entry.isDirectory()<\/code>). In that case,printDirectory()<\/code> is executed again, and the current directory depth is increased by 1. This is a nice example of a recursive call. <\/li>\r\n- Another entry is present and it is a file. In this case the file size is queried and output with
entry.size()<\/code>. After that, the while loop continues. <\/li>\r\n<\/ul>\r\n\nHere is the output:<\/p>\r\n
\n![\"Output](\"https:\/\/wolles-elektronikkiste.de\/wp-content\/uploads\/2022\/11\/output_add_remove_list_files_and_folders.png\")
<\/a>Output of add_remove_list_files_and_folders.ino<\/figcaption><\/figure><\/div>\n A few things stand out:<\/p>\r\n
- \r\n
- SYSTEM~1 is the SystemVolumeInformation directory. It is created automatically by Windows. Since its name has more than eight letters, it is abbreviated.\r\n
- \r\n
- You can delete the directory without hesitation because it is not needed for FAT16\/FAT32. However, you must first delete the two files in this directory.
SD.rmdir()<\/code> works only with empty directories.<\/li>\r\n- If you then read the SD card again with a reader on the PC, the directory and its contents will be recreated.<\/li>\r\n<\/ul>\r\n<\/li>\r\n
- The SD library converts all file and directory names to uppercase.<\/li>\r\n<\/ul>\r\n\n
add_remove_list_files_and_folders for the ESP32<\/h3>\n\n
The corresponding sketch for the ESP32 does basically the same. Here, however, the directory depth (“noOfFolderLevels”) must be specified. A value greater than the actual directory depth (which you may not know) does no harm.<\/p>\r\n
The sketch is based on parts of the example sketch SD_Test.ino. The original sketch is a little more complex than it needs to be because the SD object is also passed to the functions there. But this is not necessary. <\/p>\r\n
In addition, the sketch provides general information about the SD card, namely type and size.<\/p>\r\n\n
\r\n#include \"SD.h\"\r\n\r\nvoid setup(){\r\n int noOfFolderLevels = 3;\r\n Serial.begin(115200);\r\n delay(1000);\r\n if(!SD.begin()){\r\n Serial.println(\"Card Mount Failed\");\r\n return;\r\n }\r\n uint8_t cardType = SD.cardType();\r\n\r\n if(cardType == CARD_NONE){\r\n Serial.println(\"No SD card attached\");\r\n return;\r\n }\r\n\r\n Serial.print(\"SD Card Type: \");\r\n if(cardType == CARD_MMC){\r\n Serial.println(\"MMC\");\r\n } else if(cardType == CARD_SD){\r\n Serial.println(\"SDSC\");\r\n } else if(cardType == CARD_SDHC){\r\n Serial.println(\"SDHC\");\r\n } else {\r\n Serial.println(\"UNKNOWN\");\r\n }\r\n uint64_t cardSize = SD.cardSize() \/ (1024 * 1024);\r\n Serial.printf(\"SD Card Size: %lluMB\\n\", cardSize);\r\n\r\n Serial.println(\"Adding files:\");\r\n SD.mkdir(\"\/new_fold\"); \/\/ make new folder\r\n File dataFile = SD.open(\"\/file_1.txt\", FILE_WRITE);\r\n dataFile.println(\"Some Content\");\r\n dataFile.close();\r\n\r\n dataFile = SD.open(\"\/new_fold\/file_2.txt\", FILE_WRITE);\r\n dataFile.println(\"Some other Content\");\r\n dataFile.close();\r\n \r\n listDir(\"\/\", noOfFolderLevels);\r\n\r\n Serial.println();\r\n Serial.println(\"After removing files:\");\r\n SD.remove(\"\/file_1.txt\");\r\n SD.remove(\"\/new_fold\/file_2.txt\");\r\n SD.rmdir(\"\/new_fold\");\r\n\r\n listDir(\"\/\", noOfFolderLevels);\r\n}\r\n\r\nvoid loop(){}\r\n\r\nvoid listDir(const char * dirname, uint8_t levels){\r\n Serial.printf(\"Listing directory: \");\r\n Serial.println(dirname);\r\n \r\n File root = SD.open(dirname);\r\n if(!root){\r\n Serial.println(\"Failed to open directory\");\r\n return;\r\n }\r\n if(!root.isDirectory()){\r\n Serial.println(\"Not a directory\");\r\n return;\r\n }\r\n\r\n File file = root.openNextFile();\r\n while(file){\r\n if(file.isDirectory()){\r\n Serial.print(\" DIR : \");\r\n Serial.println(file.name());\r\n if(levels){\r\n listDir(file.name(), levels -1);\r\n }\r\n } else {\r\n Serial.print(\" FILE: \");\r\n Serial.print(file.name());\r\n Serial.print(\" SIZE: \");\r\n Serial.println(file.size());\r\n }\r\n file = root.openNextFile();\r\n }\r\n}<\/pre>\r\n<\/p>\r\n<\/div>\r\n\n
And this is what the output looks like:<\/p>\r\n\n
![\"\"](\"https:\/\/wolles-elektronikkiste.de\/wp-content\/uploads\/2022\/11\/output_add_remove_list_files_and_folders_esp32.png\")
<\/a>Output of add_remove_list_files_and_folders_esp32.ino<\/figcaption><\/figure>\n Further functions of the classes SD and File<\/h2>\n\n
Many functions classes SD and File we have already discussed. A complete listing can be found here<\/a> on the Arduino pages.<\/p>\r\n\n
![\"The](\"https:\/\/wolles-elektronikkiste.de\/wp-content\/uploads\/2022\/11\/file_sd_class_functions.png\")
<\/a>The functions of the classes SD and File<\/figcaption><\/figure>\n Testing the writing speed<\/h2>\n\n
To check the writing speed, I wrote the following sketch. It writes 1000 integer values to a file. Two variants are being tested. First, the file is opened, and the values are written “in one go” to numbersa.txt. In a second pass, the file is opened again for each write operation (to numbersb.txt).<\/p>\r\n\n
\r\n#include <SD.h>\r\nconst int chipSelect = 10;\r\n\r\nvoid setup(){\r\n String myFile = \"numbersa.txt\";\r\n Serial.begin(9600);\r\n SD.begin(chipSelect);\r\n SD.remove(myFile);\r\n unsigned long startTime = millis();\r\n\r\n \/\/ Continuous writing:\r\n File dataFile = SD.open(myFile, FILE_WRITE);\r\n for (int i=0; i<1000; i++){\r\n dataFile.println(i);\r\n }\r\n dataFile.close();\r\n \r\n unsigned long writingTime = millis() - startTime;\r\n Serial.println(\"Writing 1000 integers took: \");\r\n Serial.print(writingTime);\r\n Serial.println(\" [ms],\");\r\n Serial.println(\"when keeping file open\");\r\n Serial.println();\r\n\r\n \/\/ Discontinuous writing: n x (open, write, close)\r\n myFile = \"numbersb.txt\";\r\n SD.remove(myFile);\r\n startTime = millis();\r\n for (int i=0; i<1000; i++){\r\n dataFile = SD.open(myFile, FILE_WRITE); \/\/ FILE_APPEND for ESP32\r\n dataFile.println(i);\r\n dataFile.close();\r\n }\r\n \r\n writingTime = millis() - startTime;\r\n Serial.println(\"Writing 1000 integers took: \");\r\n Serial.print(writingTime);\r\n Serial.println(\" [ms],\");\r\n Serial.println(\"when reopening the file for every entry\");\r\n Serial.println();\r\n delay(1000);\r\n}\r\n\r\nvoid loop(){}<\/pre>\r\n<\/p>\r\n<\/div>\r\n\n
Here is the result using an Arduino Nano:<\/p>\r\n\n
![\"Output](\"https:\/\/wolles-elektronikkiste.de\/wp-content\/uploads\/2022\/12\/output_sd_card_speed_test.png\")
<\/a>Output of sd_card_speed_test.ino<\/figcaption><\/figure>\n The result speaks for itself: Opening and closing the file takes a few milliseconds each, while the actual writing of a single integer value takes place in the microsecond range. So, if speed is important, you should leave the file open.<\/p>\r\n
The file numbersa.txt is 4.77 kilobytes in size. We needed 0.266 seconds for this, which results in a write speed of 17.9 KB\/s. But wasn’t the minimum writing speed of SD cards in the range of a few MB\/s? This is true, but the bottleneck here is data processing and transmission.<\/p>\r\n
The write process on a Wemos D1 Mini board was considerably faster. It only took 57 (!) milliseconds. For an ESP32 development board, I determined 75 milliseconds.<\/p>\r\n
Moreover, the speed depends on the SD card. I determined the above values with a 16 GB microSDHC card. The time required to write the 1000 integer values, on the other hand, was 60 to 100 milliseconds higher when using a 2 GB microSD card (without “HC”).<\/p>\r\n\n
Acceleration for Arduino AVR Boards<\/h3>\n\n
If you want to write larger amounts of data, it is more effective to store them temporarily in a buffer and then transfer the buffer in one go. At least this is true when using an AVR-based Arduino board. Using this sketch (based on the nonBlockingWrite example sketch), I was able to reduce the write time for 1000 integers from 266 milliseconds to 141 milliseconds.<\/p>\r\n\n
\r\n#include <SD.h>\r\n\r\nconst char filename[] = \"demo.txt\";\r\nFile txtFile;\r\nunsigned long lastMillis = 0;\r\n\r\nvoid setup() {\r\n Serial.begin(115200);\r\n while (!Serial);\r\n\r\n \/\/ reserve memory for a String used as a buffer\r\n String buffer;\r\n buffer.reserve(288); \/\/ reserve memory for the buffer (slightly bigger than the chunk size)\r\n\r\n if (!SD.begin(10)) {\r\n Serial.println(\"Card failed, or not present\");\r\n \/\/ don't do anything more:\r\n while (1);\r\n }\r\n \/\/ If you want to start from an empty file,\r\n \/\/ uncomment the next line:\r\n SD.remove(filename);\r\n \r\n txtFile = SD.open(filename, FILE_WRITE);\r\n if (!txtFile) {\r\n Serial.print(\"error opening \");\r\n Serial.println(filename);\r\n while (1);\r\n }\r\n \r\n int counter = 0;\r\n unsigned long startingTime = millis();\r\n \r\n while(counter < 1000){\r\n fastWrite(buffer, counter);\r\n counter++;\r\n }\r\n txtFile.write(buffer.c_str()); \/\/ write what is left in the buffer\r\n unsigned long writingTime = millis() - startingTime;\r\n txtFile.close(); \r\n Serial.print(\"Writing Time [ms]: \");\r\n Serial.println(writingTime);\r\n}\r\n\r\nvoid loop() {}\r\n\r\nvoid fastWrite(String &buf, int counter){\r\n const unsigned int chunkSize = 256;\r\n char charBuf[10]; \/\/ buffer for t\r\n dtostrf(counter, 1, 0, charBuf); \/\/ convert counter char array\r\n buf += charBuf;\r\n buf += \"\\r\\n\"; \/\/ add carriage return and line feed\r\n if ((buf.length() >= chunkSize)) {\r\n txtFile.write(buf.c_str(), chunkSize); \/\/ write chunk\r\n buf.remove(0, chunkSize); \/\/ remove from buffer what's already written\r\n }\r\n}<\/pre>\r\n<\/p>\r\n<\/div>\r\n\n
The sketch is quite substantial. But to keep the article from getting too long, I’ll just explain it in broad terms:<\/p>\r\n
- \r\n
buffer<\/code> is the buffer memory for the numbers to be written. Its capacity must be slightly higher than the size of the chunks to be written.<\/li>\r\ndtostrf()<\/code> converts the number to be written into a character array which is written to the end of the buffer memory.<\/li>\r\n- If the buffer is larger than the chunk size, the buffer is written to the SD card and reduced accordingly. After that, the buffer is filled again.<\/li>\r\n
- At the end, the rest of the buffer is written to the SD card.<\/li>\r\n<\/ul>\r\n\n
For the Wemos D1 Mini Board, I could not achieve an increase in writing speed with it. It does not work on the ESP32, but further explanation would lead too far.<\/p>\r\n\n
SD cards as data loggers <\/h2>\n\n
One of the main applications for SD cards with microcontrollers is as data loggers. In the examples, you will find a simple sketch. I go one step further and control the recording of the values temporally via an alarm of a DS3231 RTC<\/a> module. This has the advantage that you can record the date and time of your measurement. Furthermore, you can send your microcontroller to sleep meanwhile and wake it up by the DS3231. Here is the basic circuit:<\/p>\r\n\n
![\"Data](\"https:\/\/wolles-elektronikkiste.de\/wp-content\/uploads\/2022\/11\/Datalogger_Wiring-1024x489.png\")
<\/a>Data logger with the DS3231<\/figcaption><\/figure>\n To measure just anything, I connected a potentiometer to the analog input A1. You can replace that with something more exciting. This is the sketch:<\/p>\r\n\n
\r\n#include <RTClib.h>\r\n#include <Wire.h>\r\n#include <SD.h>\r\n#include <avr\/sleep.h> \/\/ comment if you don't use an AVR based Board\r\nconst int chipSelectPin = 10;\r\nconst int clockInterruptPin = 2;\r\nconst int analogPin = A1;\r\nString myFile = \"data_log.csv\";\r\n\r\nRTC_DS3231 rtc;\r\n\r\nvoid setup() {\r\n Serial.begin(9600);\r\n pinMode(analogPin, INPUT);\r\n \r\n SD.begin(chipSelectPin); \r\n SD.remove(myFile); \r\n File dataFile = SD.open(myFile, FILE_WRITE);\r\n dataFile.println(\"Time;analogRead\"); \/\/ you my need a \",\" instead of \";\"\r\n dataFile.close();\r\n \r\n rtc.begin();\r\n rtc.adjust(DateTime(F(__DATE__), F(__TIME__))); \/\/ set time to system at when compiling \r\n rtc.disable32K(); \/\/disable 32K pin\r\n rtc.clearAlarm(1); \/\/ in case there's still an alarm \r\n rtc.clearAlarm(2);\r\n rtc.writeSqwPinMode(DS3231_OFF); \/\/ we use the SQW pin for interrupts\r\n rtc.disableAlarm(2); \/\/ we don't need alarm 2\r\n rtc.setAlarm1(rtc.now() + TimeSpan(10), DS3231_A1_Second); \/\/ alarm in 10 seconds\r\n\r\n pinMode(clockInterruptPin, INPUT_PULLUP);\r\n attachInterrupt(digitalPinToInterrupt(clockInterruptPin), onAlarm, FALLING);\r\n}\r\n\r\nvoid loop() {\r\n if(rtc.alarmFired(1)) {\r\n unsigned int measuredValue = analogRead(analogPin);\r\n char date[18] = \"DD.MM.YY hh:mm:ss\";\r\n rtc.now().toString(date);\r\n rtc.clearAlarm(1);\r\n \r\n Serial.println(date);\r\n Serial.print(\"Analog Read: \"); \r\n Serial.println(measuredValue);\r\n File dataFile = SD.open(myFile, FILE_WRITE); \/\/ FILE_APPEND for ESP32\r\n dataFile.print(date);\r\n dataFile.print(\";\");\r\n dataFile.println(measuredValue);\r\n dataFile.close();\r\n rtc.setAlarm1(rtc.now() + TimeSpan(10),DS3231_A1_Second);\r\n delay(100); \/\/ give some time for Serial.print() \/ Data.print() to complete\r\n \r\n set_sleep_mode(SLEEP_MODE_PWR_DOWN); \/\/ choose power down mode, comment if you don't use an AVR based Board\r\n sleep_mode(); \/\/ sleep now! \/\/ comment if you don't use an AVR based Board\r\n } \r\n}\r\n\r\nvoid onAlarm() {\r\n Serial.print(\"Alarm at: \");\r\n}<\/pre>\r\n<\/p>\r\n<\/div>\r\n\n
Explanations for data_logger.ino<\/h4>\n\n
In order not to make the article too long, I will not go into the details of the DS3231 control. Please have a look at this article<\/a>. In broad terms, here’s what the sketch does:<\/p>\r\n
- \r\n
- Every 10 seconds, the DS3231 triggers an alarm.<\/li>\r\n
- The alarm wakes up the Arduino, which then takes a measurement.<\/li>\r\n
- The measured value is written to the SD card in the data_log.csv file, together with the date and time. The format is: “DD.MM.YY hh:mm:ss;measured value”.<\/li>\r\n
- The first line in data_log.csv is “Time;analogRead” and serves as a header.<\/li>\r\n
- Measured value and time are also output on the serial monitor, which is only for control purposes.<\/li>\r\n
- The Arduino is sent to sleep and everything starts over again.<\/li>\r\n<\/ul>\r\n
- You can delete the directory without hesitation because it is not needed for FAT16\/FAT32. However, you must first delete the two files in this directory.
- Another entry is available and it is a directory (
- To read out the data as an integer, the practical function
- There are two options for reading the data:\r\n
- So “myFile” contains only the filename or the path to the file. In contrast, “dataFile” is the actual file object.\r\n
- For somewhat larger amounts of data up to ~1 megabyte you can use external EEPROMs. These are characterized by low space and power requirements. I described external, I2C-controlled EEPROMs here<\/a>. An article about the faster, SPI-based representatives can be found here<\/a>.<\/li>\r\n
- SD cards as data logger<\/a><\/li>\r\n
![\"Output](\"https:\/\/wolles-elektronikkiste.de\/wp-content\/uploads\/2022\/11\/datalogger_serrial_monitor.png\")
<\/a>![\"data_log.csv](\"https:\/\/wolles-elektronikkiste.de\/wp-content\/uploads\/2022\/11\/datalogger_raw_in_excel.png\")
<\/a>![\"Display](\"https:\/\/wolles-elektronikkiste.de\/wp-content\/uploads\/2022\/11\/datalogger_in_excel-1024x377.png\")
<\/a>