Once in a while though, something gets into a funk, and the setup stops working. Power off, power on, still the same. Without touching any of the connections at all, left it powered off for multiple days, and then on powering up again, it just starts working again. What would cause this?

I’m relatively new to the world of Arduino and microcontrollers and trying to understand how to setup several sdi12 sensors that measure stream depth (pressure transducers) and one modbus meter in a pipe. The sensor data must be reported to the web using cellular data. The system must be solar powered.in case anyone is wondering this will be used in California to monitor water diverted from a river for agriculture. This is a requirement by the state water board for surface water diversions over a size (I forget the threshold)

That’s the end goal, but I think it’s more realistic to start with a simplified version that includes a single sdi12 pressure transducers. There is a lot of information on this website, but I haven’t seen an overview of how a full setup would be assemble.

questions:

is the LiuDR SDI-12 USB adapter appropriate for this application?

where is the best place to start researching something like this? Is there a beginners guide in the blog somewhere?

Do you have a description of a similar system that I can read about?

Thank you!

in case anyone is interested in my role, I work for a nonprofit that works with agricultural water users to apply management improvements that improve ecological function without hurting production. I use a lot of pressure transducers, and soil moisture sensors. Pulling data from the sensors takes time and sometimes results in error and lost data when loggers are not redeployed properly. I’d like to move toward a fully telemetered system as old sensors need replacement.

I'm interested in your SDI-12 adapter to connect multiple environmental sensors (turbidity, conductivity, oxygen).
My goal is to build an autonomous, battery-powered system (using either LiSOCl₂ or Li-Po batteries) with at least one month of autonomy.

I’m not sure which kind of "computer" or microcontroller would be best suited for your system, since my only requirement is to store data locally on an SD card.
Do you have any recommendations for a compact and low-power platform?
I’m considering options like the Raspberry Pi Zero or a STM32-based board, but low power consumption and small size are essential since the probe will be deployed at sea.

Also, my sensors draw brief current peaks of 500 mA for 2 ms at 12 V.
Do you think your SDI-12 adapter can handle this type of load?

Thank you in advance for your help — and for your great work.

Best regards,
Julia

Hey there!
I’ve got a quick question for you hoping for some guidance after purchasing your raspberry pi sdi-12 usb adapter a while back.

I can’t seem to get the pi to recognize the sensor I’m trying to use and was wondering if you could confirm its compatibility. It may be that this sensor sends data in an unrecognized format or something. The Pi does recognize the usb device.

My setup: RPi 5, your sdi adapter, connected with a 9V external power source to power this sensor: https://www.apogeeinstruments.com/sil-411-commercial-grade-sdi-12-digital-output-standard-field-of-view-infrared-radiometer-sensor/

The specs are available on that page. Thanks for your time, it’s greatly appreciated!

I would like to connect 6 Terros 12 sensors with a Raspberry pi 4 and want to know which product is most suitable for this, and what extra parts I may need.

Also, do you have an estimate of how long the shipment may take?

Thank you for the help and support!

Alexis

In general, all SDI-12 sensors should work with my adapters. But there are always some variations in the implementations of the SDI-12 standard by different vendors. Here is one brand of sensors that was recently confirmed working with my adapter. Thanks Michael!

According to the website below, FloraPulse sensors are "the most accurate irrigation guidance for orchards and vineyards".

https://www.florapulse.com/

The only slight modification that Michael found was that a delay of 1 second is needed before sending each command to the adapter. You can delay with time.sleep(1)

Here is a snippet courtesy of Dr. Michael Santiago:

#!/usr/local/opt/python-3.5.1/bin/python3.5
# Simple SDI-12 Sensor Reader Copyright Dr. John Liu
import serial.tools.list_ports
import serial
import time
import re

ser=serial.Serial(port='COM10',baudrate=9600,timeout=10)
time.sleep(2.5) # delay for arduino bootloader and the 1 second delay of the adapter.

ser.write(b'?!')
sdi_12_line=ser.readline()
sdi_12_line=sdi_12_line # remove \r and \n since $ has trouble with \r
m=re.search(b'$',sdi_12_line) # having trouble with the \r
sdi_12_address=m.group(0) # find address
print('\nSensor address:', sdi_12_address.decode('utf-8'))

time.sleep(1)
ser.write(sdi_12_address+b'I!')
sdi_12_line=ser.readline()
sdi_12_line=sdi_12_line # remove \r and \n
print('Sensor info:',sdi_12_line.decode('utf-8'))

time.sleep(1)
ser.write(sdi_12_address+b'M!')
sdi_12_line=ser.readline()
sdi_12_line=ser.readline()

time.sleep(1)
ser.write(sdi_12_address+b'D0!')
sdi_12_line=ser.readline()
sdi_12_line=sdi_12_line # remove \r and \n
print('Sensor reading:',sdi_12_line.decode('utf-8'))
print('\nFor complete data logging solution, download the free Python data logger under "data logger programs"\nhttps://liudr.wordpress.com/gadget/sdi-12-usb-adapter/')
ser.close()

If you are planning to build your own data logger with free time-series database such as influxdb, this will be a good start to read data.

1. Log data from the Meter Group devices Terros12 and Atmos14; accommodating the Atmos14 need for a delay between querying the device and requesting a reading.
2. Modifying the script so the Thingspeak channel is defined by the device in the user configuration to allow devices to send to independent channels. 

Updating the config file to point to multiple Thingspeak API destinations:

edit the api_key field of the config file in notepad

from  "api_key" : "some_api_key" to
 "api_key": {"sensor_address_1": api_key_for_sensor1", "sensor_address_2": api_key_for_sensor2"  }

seprate sensors with comma
e.g: 
 "api_key": {"1" : "GTOEBKK8ZQHI1V1B" , "2" : "XXXXXXXXXXXXXXXX" }

Example Config aka Liudrlogger.conf file when pointing to multiple channels:

{"wifi_ssid": "datalogger", "wifi_passwd": "logger1234", "my_timezone": "CET-1CEST", "channelID": "359964", "api_key":{"1":"GTOEBKK8ZQHI1V1B" , "2" : "SECONDAPIKEYHERE" }, "total_data_count": 1, "delay_between_pts": 30, "sdi_12_address": "12", "sdi_12_command": , "analog_inputs": "N", "time_zone_choice": 1, "ser": , "ser_ID": []}

And finally the revised 1.5.7 logger script:

import datetime  # For finding system's real time
import json  # For serializing and deserializing parameters on configuration file.
import re  # For regular expression support
import serial.tools.list_ports  # For listing available serial ports
import serial  # For serial communication
import signal  # For trapping ctrl-c or SIGINT
import sys  # For exiting program with exit code
import time  # For delaying in seconds
import urllib.parse  # For encoding data to be url safe.
import urllib.request  # send data to online server
# import os # For running command line commands
# import platform # For detecting operating system flavor
"""
SDI-12 Sensor Data Logger Copyright Dr. John Liu
2022-11-25 Revision by Grow_Deluxe to allow the script to log data from the Atmos 14/Terros 12 by Meter Group. In addition this revision writes to a .csv file per device, and uploads each device to a unique thingspeak channel api (see readme.txt for more details on this)
2021-10-25 Updated to up to 8 fields for thingspeak and updated API address
2018-07-03 Improved exception handling for the SDI-12 protocol. No response from the sensor will not trigger exception.
    Rather, the loop waits for the next iteration and try again. Time to read line 274
    Added opening port by ID feature in parameters and saving port ID feature in interactive session.
2018-04-24 Added exception handling for opening a non-existing serial port (possibly the config file has wrong port name).
    Added exception handling for http.client.BadStatusLine from urllib calls
2018-04-21 Tested recently added features such as D0, D1, and M, M1.
2018-04-19 Updated the script to issue multiple commands such as M and M1.
    Added features to collect all data using D0, D1, etc. until it collects all measurements
    No longer asks for analog sensors. Just type in address z with the other sensors and issue
    commands 0 or 1 to collect single-ended or differential analog channels from SDI-12 + Analog adapter.
    The script saves a configuration file Liudrlogger.conf. You can modify it with a text editor.
    It's very easy to understand. With the config file, the logger starts in auto-logging mode.
    If you delete the file, it starts interactive session to gather the parameters from the user.
2018-04-07 Replaced capitalize() with upper. Added urllib.error.URLError to exception handling.
    Commented out os and platform imports and unit_id.
    Decoded byte strings before converting into float to stay compatible with MicroPython.
2018-03-29 Added exception handling for urllib.request.urlopen for server internal error
2018-03-28 Replaced cURL with urllib.request for sending data to thingspeak.com server
2017-11-06 Updated telemetry code to upload to thingspeak.com from data.sparkfun.com.
2017-06-23 Added exception handling in case the SDI-12 + GPS USB adapter doesn't return any data (no GPS lock).
    Added serial port and file closing in ctrl + C handler.
2017-02-02 Added multiple-sensor support. Just type in multiple sensor addresses when asked for addresses.
    Changed sdi_12_address into regular string from byte string.
    I found out that byte strings when iterated over becomes integers.
    It's easy to cast each single character string into byte string with .encode() when needed as address.
    Removed specific analog input code and added the adapter address to the address string instead.
2016-11-12 Added support for analog inputs
2016-07-01 Added .strip() to remove \r from input files typed in windows
    Added Ctrl-C handler
    Added sort of serial port placing FTDI at item 0 if it exists
"""
rev_date = '2021-10-25'
version = '1.5.7'

config_file_name = 'Liudrlogger.conf'
# Default values of operating parameters
paras_default = dict([
    ('wifi_ssid', 'datalogger'),
    ('wifi_passwd', 'logger1234'),
    ('my_timezone', 'CET-1CEST'),
    ('channelID', '359964'),
    ('api_key', 'GTOEBKK8ZQHI1V1B'),
    ('total_data_count', 1000),
    ('delay_between_pts', 60),
    ('sdi_12_address', 'z'),
    ('sdi_12_command', ),
    ('analog_inputs', 'N'),
    ('time_zone_choice', 0),
    ('ser', []),
    ('ser_ID',[])
])


def SIGINT_handler(signal, frame):
    ser.close()
    data_file.close()
    print('Quitting program!')
    sys.exit(0)


signal.signal(signal.SIGINT, SIGINT_handler)
ser = []  # This list stores opened serial port
fnf = False  # File not found error

# This one for everyone to perform tests with
# channelID = "359964"
# api_key = "GTOEBKK8ZQHI1V1B"

# Use computer name as unit_id. For a raspberry pi, change its name from raspberrypi to something else to avoid confusion
# unit_id=platform.node()

http_request_url_format = 'https://api.thingspeak.com/update.json?api_key=%s%s'
max_upload_values = 4  # Maximal values to upload as a single data point
adapter_sdi_12_address = 'z'
# This is the flag to break out of the inner loops and continue the next data point loop in case no data is received from a sensor such as the GPS.
no_data = False
ports = []
VID_FTDI = 0x0403;


def load_paras():
# Load operating parameters from default dictionary.    
# The function returns a dictionary, which has keys for all parameters needed in the program.
    try:
        f = open(config_file_name, 'r')
        par = json.load(f)
        f.close()
        return (par, False)  # Return paras and False for file not found
    except FileNotFoundError as e:
        print('No configuration file found. Starting interactive session...')
        return (paras_default, True)  # Return default paras and True for file not found


def print_credit(pa):
    print('+-' * 40)
    print('SDI-12 Sensor and Analog Sensor Python Data Logger with Telemetry V', version)
    print(
        'Designed for Dr. Liu\'s family of SDI-12 USB adapters (standard,analog,GPS)\n\tDr. John Liu Saint Cloud MN USA',
        rev_date, '\n\t\tFree software GNU GPL V3.0')
    print('\nCompatible with PCs running Win 7/10, GNU/Linux, Mac OSX, Raspberry PI, Beagle Bone Black')
    print('\nThis program requires Python 3.4, Pyserial 3.0, and internet connector (data upload)')
    print('\nUsing config file:%s' %(config_file_name))
    print('\nData is logged to YYYYMMDD.CVS in the Python code\'s folder')
    print('\nVisit https://thingspeak.com/channels/%s to inspect or retrieve data' % (pa))
    print('\nFor assistance with customization, telemetry etc., contact Dr. Liu.')
    print('\nhttps://liudr.wordpress.com/gadget/sdi-12-usb-adapter/')
    print('+-' * 40)


def interactive_session(pa):
    # List ports for user to select
    a = serial.tools.list_ports.comports()
    for w in a:
        ports.append((w.vid, w.device, w.serial_number))

    ports.sort(key=lambda ports: ports)

    print('\nDetected the following serial ports:')
    i = 0
    for w in ports:
        print('%d)\t%s\t(USB VID=%04X)\tID#:=%s' % (i, w, w if (type(w) is int) else 0,w))
        i = i + 1
    total_ports = i  # now i= total ports

    user_port_selection = input('\nSelect port from list (0,1,2...). SDI-12 adapter has USB VID=0403:')
    if (int(user_port_selection) >= total_ports):
        exit(1)  # port selection out of range

    # Open serial port
    pa.append((ports))
    ser.append(serial.Serial(port=(ports), baudrate=9600, timeout=10))
    time.sleep(2.5)  # delay for arduino bootloader and the 1 second delay of the adapter.

    pa = int(input('Total number of data points:'))
    pa = int(input('Delay between data points (second):'))
    # Enter addresses and commands for each sensor
    pa = ''
    user_sdi_12_address = input(
        'Enter all SDI-12 sensor addresses as a list, such as 123z.\nIf you have analog channels on your adapter, include address z to your list:')
    pa = user_sdi_12_address.strip()  # Remove any \r from an input file typed in windows
    pa = []  # Clear the list of commands
    print('\n\nSensor measurement commands:\nBasic sensors respond to M0 (same as M) command.')
    print(
        'More sophiscated sensors respond to more commands, such as M0 for moisture, and M1 for temperature for an HSTI soil probe.')
    print('\nTo send the basic command M0, enter 0. To send more commands such as M0, M1 and M4, enter 014.')
    for addr in pa:  # Get measurement commands
        print('\nFor SDI-12 sensor %c, which command(s) should be sent?' % (addr))
        pa.append(input('Enter command:'))

    # pa=input('Collect analog inputs (requires SDI12-USB + Analog adapter)? (Y/N)')
    # pa=(pa.strip()).upper() # Remove any \r from an input file typed in windows and capitalize answer
    print('Time stamps are generated with:\n0) GMT/UTC\n1) Local\n')
    pa = int(input('Select time zone.'))
    f = open(config_file_name, 'w')  # Save settings
    json.dump(paras, f)
    f.close()


# if len(sdi_12_address)==0:
#    sdi_12_address=adapter_sdi_12_address # Use default address


def sensor_info(pa):
    for an_address in pa:
        ser.write(an_address.encode() + b'I!')
        sdi_12_line = ser.readline()
        print('Sensor address:', an_address, ' Sensor info:', sdi_12_line.decode('utf-8').strip())


(paras, fnf) = load_paras()
print_credit(paras)

if (fnf):  # No configuration file. Start interactive session. Serial port is open in the interactive session.
    interactive_session(paras)
else:  # Open serial port
    print('\nUsing saved configuration...\n')
    try:
        ser.append(serial.Serial(port=paras, baudrate=9600, timeout=10))
        time.sleep(2.5)  # Wait for arduino bootloader
    except serial.serialutil.SerialException as err:
        print(err.__str__())
        print('Invalid serial port in config file!')
        sys.exit(1)

sensor_info(paras)

# Open data file for logging
if paras == 0:
    now = datetime.datetime.utcnow()  # use UTC time instead of local time
elif paras == 1:
    now = datetime.datetime.now()  # use local time, not recommended for multiple data loggers in different time zones

ser_ptr = 0

while True:
    
    if paras == 0:
        now = datetime.datetime.utcnow()
    elif paras == 1:
        now = datetime.datetime.now()
      # formatting date and time
    for (cmd_ptr, an_address) in enumerate(paras):
        i = 0  # This counts to max_upload_values to limit data sent to the server.
        value_str = ''  # This stores &value0=xxx&value1=xxx&value2=xxx&value3=xxx&value4=xxx&value5=xxx and is only reset after all sensors are read.

        output_str = "%04d/%02d/%02d %02d:%02d:%02d%s" % (now.year, now.month, now.day, now.hour, now.minute, now.second,' GMT' if paras == 0 else '')
        
        # get device info & create file name with device info
        ser.write(an_address.encode() + b'I!')
        sdi_12_line = ser.readline()
        
        file_name= an_address+ (sdi_12_line.decode('utf-8').strip()).replace(" ","") + ".csv";
        
        # print("filename : " , repr(file_name))   
        
        
        data_file_name = file_name
        data_file = open(data_file_name, 'a')  # open yyyymmdd.csv for appending
        print('Saving to %s' % data_file_name)
        values = []  # clear before each sensor
        sdi_12_line_buffer = b''  # This stores all data from the same sensor address, including from M!->D0! D1!, M1!->D0!, D1! etc.
        for a_command in paras:
            try:
                if a_command == '0':
                    complete_command = an_address.encode() + b'M!'
                else:
                    complete_command = an_address.encode() + b'M' + a_command.encode() + b'!'
                ser.write(complete_command);  # start the SDI-12 sensor measurement
                # print(complete_command);  # start the SDI-12 sensor measurement
                sdi_12_line = ser.readline()
                # print(sdi_12_line)
                if sdi_12_line==b'': # Didn't get a response from the sensor? Faulty wiring?
                    print('Sensor %s failed to respond to command %s.' %(an_address,complete_command))
                    no_data = True # End the current iteration of sensors and commands on each sensor and wait for the next iteration.
                    break;
                # print("aM! response ", repr(sdi_12_line))   
                sdi_12_line = sdi_12_line  # remove \r and \n since $ has trouble with \r
                m = re.search(b'$', sdi_12_line)  # This should match a number () that appears at the end of the response ($), which is a 1-digit number of "returned values" but it is having trouble with the \r so I removed \r\n in the previous line of code.
                if (not m):  # Match evaluates into True. The response is wrong. There should be a number at the end of the response, save \r\n, but it is not in this response.
                    no_data = True # End the current iteration of sensors and commands on each sensor and wait for the next iteration.
                    break;
                total_returned_values = int(m.group(0))  # find out how many values are returned
                warm_up_time = int((sdi_12_line.decode())); #extract ttt from atttn, removing address(a) and no. of return values(n) 
                # print(total_returned_values)

                if warm_up_time >0 :                    
                    # print("warmup time for sensor %s: " % (an_address) , sdi_12_line.decode() )
                    time.sleep(warm_up_time); # wait for warm-up time
                    sdi_12_line = ser.readline()  # read the service request line
                    
                    # print("<cr><lf> : " ,repr(sdi_12_line))
                    if sdi_12_line != an_address.encode() + b'\r\n':
                        print('Sensor %s didn\'t respond with correct service request.' % (an_address))
                        no_data = True  # End the current iteration of sensors and commands on each sensor and wait for the next iteration.
                        break;
                    # Read as much data as you can with D0, D1, ... D9 until only the address and \r\n is returned
                for d_command in range(10):
                    complete_command = an_address.encode() + b'D' + str(d_command).encode() + b'!'
                    ser.write(complete_command)  # request data
                    # print(complete_command)  # request data
                    next_sdi_12_line = ser.readline()  # read the data line
                    # print(sdi_12_line)
                    if (len(next_sdi_12_line) <= 3): # only 1\r\n is returned, indicating that the sensor has run out of values to return. It's time to stop asking.
                        break
                    else:
                        next_sdi_12_line = next_sdi_12_line  # remove address, \r and \n since $ has trouble with \r, stitch all responses from D0! to D9! together for later processing.
                        sdi_12_line_buffer += next_sdi_12_line  # Append results from the Dn! command to data from D0! to Dn-1!
            except serial.serialutil.SerialException as err:
                print(err.__str__())
                ser.close()
                sys.exit(1)
            # print(sdi_12_line_buffer) # Received data from one address one M command
            for iterator in range(total_returned_values):  # extract the returned values from SDI-12 sensor and append to values[]
                m = re.search(b'+', sdi_12_line_buffer)  # search a number string with preceding + or - sign and any number of digits and decimal (+).
                try:  # if values found is less than values indicated by return from M, report no data found. This is a simple solution to GPS sensors before they acquire lock. For sensors that have lots of values to return, you need to find a better solution.
                    values.append(float(m.group(0).decode()))  # convert into a number. Decode byte string into string first due to MicroPython.
                    sdi_12_line_buffer = sdi_12_line_buffer
                except AttributeError:
                    print("No data received from sensor at address %c\n" % (an_address))
                    time.sleep(paras)
                    no_data = True
                    break
            if (no_data == True):
                break;                  
        
        
        output_str = output_str + ',' + an_address

        for value_i in values:
            output_str = output_str + ",%s" % (value_i)  # Output returned values
            if (i < max_upload_values):
                value_str = value_str + "&field%d=%s" % (i + 1, value_i)  # format values for posting. Field starts with field1, not field0.
                i = i + 1
                
        print(output_str)
        output_str = output_str + '\n'
        data_file.write(output_str)
               
        
        if (no_data == True):
            no_data = False
            time.sleep(paras)
            continue;
        while (i < max_upload_values):  # Pad with zeros in case we don't have max_upload_values fields. This is only necessary for certain servers.
            value_str = value_str + "&field%d=0" % (i + 1)  # format values for posting. Field starts with field1, not field0.
            i = i + 1
        
        http_request_url = http_request_url_format % (paras, value_str)  # Format url command
        print(http_request_url)  # Debug information
        try:
            req = urllib.request.urlopen(http_request_url)
        except: # Intermittent internet connection could cause more underlying modules to through exceptions. Just catch any exception and discard.
            print("Unexpected error:", sys.exc_info())
        #except (urllib.error.HTTPError, urllib.error.URLError, http.client.BadStatusLine) as err:
            #print('Error uploading data.')
            #print(err.__str__())  # Sometimes the server returns with 500 Internal error and this error is raised and needs to be caught otherwise it breaks the script.
            # You can decide whether to send the request one or a few more times or just discard the error and move on.
        else:
            print('Sent data')       
            
        values = []  # clear values for the next iteration, 3.2.3 doesn't support clear as 3.4.3 and 3.5.1 does
        data_file.flush()  # make sure data is written to the disk so stopping the scrit with ctrl - C will not cause data loss
    
    #print(req.status)  # Send data to server and print out response. 200 means OK.

    
    time.sleep(paras)
ser.close()

1. One printed circuit board with many 0.1" spacing holes

1. One row of 40-oin male break-away headers

1. One small cut sheet of blu-tack putty to hold parts to be soldered (I used a blob in the video but I'll supply fresh ones cut from a sheet)

Here is a play list of how to solder on a printed circuit board:

https://www.youtube.com/playlist?list=PLwsP5zYBAWdmDGuesC7Rmr-RW3JxSFtAs

The kit is now for sale on my square store:

https://liudr.square.site/product/soldering-practice-kit/74?cs=true&cst=custom