If you’ve been following my posts, I discussed how to use my SDI-12 USB adapter and raspberry pi (or PC) to log data from SDI-12 sensors. Now that you have your data, you can also send them to sparkfun’s data server for storage and later retrieval. In the following video, I explained what you can do with this feature included in my data logger python script. I also constructed a sample webpage to display data saved on the server. Lots of customization can be made based on this sample webpage.
On my last post, I showed a couple of videos of how to connect an SDI-12 sensor to the SDI-12 USB adapter. Here is a video of how to configure an SDI-12 sensor and log data with the adapter:
Sorry there was some noise in the background that I couldn’t get rid off with the authoring software. Here is the transcript in case you need it. I will explain how to send data to sparkfun’s data server in my next post.
Let’s plug in the SDI-12 USB adapter. Windows will automatically install its driver and create a COM port, in this case, COM18. If this is your first time installing a COM port on your computer, you will need to be patient.
What we are looking at are two windows. The window on the left is the python shell. It shows you the input and output of python scripts. The window on the right side is the SDI-12 configuration script version 1.1. We run this script to set up the sensor’s SDI-12 address.
On the left window, you can see the messages printed by the configuration script in blue. This script runs on windows, mac, linux, and raspberry pi.
The script has discovered a number of COM ports and listed them all on the left window. The first one, item zero, is our SDI-12 USB adapter, i.e. COM18. Type zero and enter. Ignore the rest. If you are unsure about the COM port number, run this script with the adapter disconnected and then run it again with the adapter connected. This way you can see which port belongs to the adapter.
After a short moment, the script has detected the SDI-12 sensor at address 1. The information printed out indicates sensor address 1, compliant with SDI-12 standard version 1.3, and the manufacturer is Decagon. The sensor is a spectral reflectance sensor and its serial number is also printed out.
What this script does is to detect the one-character SDI-12 sensor address, print out the sensor information, and allows you to change its SDI-12 sensor address. Valid address includes 0-9, A-Z, and a-z.
Let’s set the sensor address to 2. To check that this has taken effect, we run the script again.
Now it has detected the new address. The address is saved on the sensor until it is changed again.
Now let’s look at the data logger script.
What this script does is: it logs data to two places, a file on the computer, and it also sends the same data to a server at sparkfun electronics. Everyone that runs this sample data logging script shares the same storage on the server and can see results from everyone else. You can also create your own storage or stream on sparkfun so you can keep the data to yourself.
Here I have python shell and sdi_12_logger script version 1.1. Let’s run the script with F5. This time there is a longer printout and it does say it runs on Mac OSX.
Next we see a list of serial ports. We select zero for COM18, like before.
Then we are required to enter the total number of data points. Let’s try 5. You can enter any large number of data points and stop the data acquisition anytime with Ctrl-C. Data saved on your computer and sent to the server will not be lost if you stop the script.
Delay between data points is specified in seconds. We’ll use 10 seconds. Then we decide whether to store each data point with local or universal time. Enter 1 to pick local time.
Enter the SDI-12 sensor address, 2.
Here is the first data point. Date and time, then two spectral reflectance values, and then 2.0 for facing upward.
The curl command that appears on the next line will only appear if you have curl installed, which is a tool to send HTTP requests. So besides a local copy of your data, you also have an online copy on sparkfun electronics server.
OK now we finished collecting 5 data points and the python shell prompt has returned.
We will take a look at the data file and the server data next.
If you wonder how to connect your SDI-12 sensor to the SDI-12 USB adapter, here are two videos showing you how to. In the video, I was using a Decagon PRI spectral reflectance sensor. It comes with 3 wire leads. Bare wire is ground. Wire wire is power. Red wire is SDI-12 bus. If you purchased a Decagon sensor with the 1/4″ stereo plug, you can purchase an adapter from them that break out the wires into 3 wire leads.
The jeweler’s screw driver that I used was 0.1″ wide flat-head. On my next post, I will demonstrate how to run configuration and data logging scripts.
Last time I described how to install python on Debian or Raspberry Pi. If you are a beginner and have a windows PC, here is my short tutorial on how to install python AND pyserial on a windows PC. Since I’m interested in Arduino, I will certainly need a serial module in python to talk to Arduino, such as sending Real time (yyyy/mm/dd hh:mm:ss or epoch) to Arduino to set its Real Time Clock or collect data from Arduino.
Python as of the writing has two main versions, Python 2.x and 3.x. The latest 2.x is 2.7.11. Lots of code have been developed on Python 2.x so many people still hang on to this version. The other day I tinkered with a Raspberry Pi LCD hat (add-on) and the Adafruit Python module that drives it can only be installed on Python 2.7.11. The most current version of Python 3 is 3.5.1. We will focus on this version since it has been out for quite a while. Also, Pyserial works flawlessly on Python 3.5.1.
Now you should have Python installed and ready to go. Still, you will need to install various modules to expand your Python, or simply to run “that script that wants this and that module”. Lots of modules are available to Python such as pyserial (serial port support) and idlex (more customization to Python’s IDE).
To install pyserial (or any other module), first open a command prompt. Then type in:
“pip install pyserial”
The package will install automatically. You are done!
Since I was going away for a month, I decided, before I left, to set my SDI-12 data logger to run a 30-day data collection routine, just to see how robust the software is. Here is the result:
The temperature sensor was in my office so its variation was small. The steep slope towards the end was because I opened the window after I got back and kept the window open for a whole day. You may notice the lack of variation of temperature between 5/10 and 5/13 but that WAS what happened. History data from wunderground.com showed less-than-average variation of outdoor temperature in my area during the same period:
So my conclusion was that my python data logger code running on raspberry pi was robust enough for at least 30 days so was the SDI-12 USB adapter. If this were done outdoors, then the raspberry pi and the SID-12 USB adapter both need protection from the element. Also a solar panel and battery will be needed.
Click PATH and Edit. Add the following TWO paths:
