2019 summer run update

July 18, 2019 Leave a comment

The 2019 summer SDI-12 USB adapter data logging run has been relatively smoothly but there were a few issues that are worth mentioning.

1. Battery

The battery that I recommended as a backup battery for raspberry pi (see photo below) needs to be used with some consideration:

I have not tested how long the charge lasts when it is powering a raspberry pi 3B (I used a raspberry pi zero last year but had to use some USB OTG adapter and a USB hub with Ethernet dongle etc. so I decided to throw my 3B in since I have a 3B+ now). What I thought should happen is that the battery will eventually drain completely after hours of power outage but once power is back on, it should turn on the raspberry pi right away. That turned out to be not the case. The battery seems to need about half an hour of charge time after it has been drained before it can power the raspberry pi 3B with enough stability that it would run normally. Within the first half hours of initial charging after a power outage, the raspberry pi was not stable and I couldn’t log in to it. The logging script didn’t run either. I ended up unplugging the pi and let the battery charge for 30 minutes and plugged the pi back in.

In case you want to prevent this issue, which may or may not affect the integrity of the pi’s operating system, you need to buy a backup battery with proper capacity. Mine is only 3000mAh, since I have the logger in my garage, only expecting at most hours of power outage:

https://www.amazon.com/TalentCell-Rechargeable-Amplifier-Multi-led-Indicator/dp/B00MHNQIR2/

There are 6000mAh and 12000mAh versions that will definitely address longer periods of power outage. If I was only expecting up to a few hours of power outage, then how come my logger drained the battery, twice?

The issue was actually with the power outlet it was plugged in, which is protected by an upstream Ground Fault Circuit Interrupter (GFCI) outlet. When I bought the house, there was no GFCI outlet in the garage! I wonder how those previous owners never thought about installing one. So I replaced the outlet in the garage with a GFCI outlet, which protects all downstream outlets. The outlet my logger is plugged in is downstream of the GFCI outlet thus is protected against ground fault, which means shorting the hot with the ground. It was the right move but unfortunately not enough. Outside my garage there is an outdoor outlet with some “outdoorish” cover. We had some VERY heavy downpours in the past month.

As you can see on the plot below, the two high spikes on soil dielectric constant correspond to two heavy rain falls:

The vertical blue lines simply indicate how quickly the rain fall caused soil’s dielectric constant to first rise then later drop (exponentially). The two almost horizontal blue lines immediately following the spikes were periods of power loss. I didn’t check my online plots and was surprised to find that the plot stopped. I guess I could use the spikes to estimate my battery’s run time to be between 4 hours and 13 hours, probably 4 hours. The following soil temperature plot during the same period shows the power outages more clearly:

So what caused the GFCI outlet to trip? It turned out to be the outdoor outlet. I took it apart. Here is what it looks like:

The gasket on the left wasn’t doing enough to protect the outlet. You can see the outlet’s rusted top side (the outlet was mounted horizontally). This must have been caused by rain water seeping into the outlet, causing ground fault.

I do recall a couple of times after bad weather our garage lost power (circuit breaker). That must have been before I installed the GFCI outlet. So in order to address the real issue that the outdoor outlet is causing, I purchased a better cover and installed a separate GFCI outlet in the power box. To be honest, the brick veneer didn’t help. It was harder to seal against due to its rough surface but the new cover with a new gasket hopefully will work better. I’ll later apply some ready-to-use cement on the top of the cover to seal any possible leaks. With this new GFCI outlet, hopefully even when it is shorted, it would act before the upstream one acts and prevents the logger from losing power for extended time again. Here is how it looks:

Next time I will write about my experience running the logging script automatically with the latest raspbian distribution.

Filed under data logging, Raspberry pi

Raspberry pi data logging sd card image updated

February 7, 2019 Leave a comment

To help people getting started using raspberry pi to log data with my SDI-12 USB adapters, I’ve been offering ready-to-go MicroSD cards with raspberry pi system image. You can pop it into your raspberry pi and get started with all the necessary tools installed, including the python logging script. Every few months, I update the image so you get the most recent operating system and the python logging script. This time I updated the image in late December. I just recently tested it on a raspberry pi 3A+ model and it works. I guess it’s no surprise. My image works on 3B and 3B+, Zero and ZeroW. The 3A+ has the same processor as 3B+ so it worked right away. I remember when Zero came out, the image I had didn’t work on it since it has a different processor than 3B (latest model then). I had to put the image in a 3B and run updates. After that it worked. It’s been a while since the raspberry pi folks designed the 3B model. I wonder if they are ready to release a model 4B later this year. I am hoping to see better performance but also hoping to see programmatic ways to dial back the performance for battery operation so the pi can enter a low-power mode while collecting data and spring back to full-throttle if it needs to crunch data (program controls power mode) or handle a user remote login (user select power mode). Anyway, if you’re stuck with an sd card having a working older image, all you have to do is to update:

[code language=”bash”]
sudo apt-get update
sudo apt-get upgrade
[/code]

In case you need more details regarding updating your raspberry pi, here is the official post:

https://www.raspberrypi.org/documentation/raspbian/updating.md

 

Filed under data logging, Raspberry pi

Raspberry pi boards comparison

February 3, 2019 2 Comments

In a previous post, I explained the most recent model of raspberry pi board, the 3B+ board, and my take on why you want a raspberry pi. This post is about comparing the different models. Again, if you are taking on the raspberry pi hobby, get the 3B+ with quad-core 1.4GHz processor, 1GB memory, and most recent Wi-Fi/Bluetooth on board. It’s easy and fun to use. Other boards are made for specific reasons or tasks.

The following is the most popular one of all, the model 3B+:

Next, let’s see the most recent model, the model 3A+:

If you compare them side by side, you’ll notice that 3A+ is almost the same as the 3A+ except it is missing the stuff on top of the photo:

Here is zero W. It is only about a third of the size of the 3B+ or half the size of the 3A+:

How do these boards compare?

A VS. B VS. Zero

There are three models of raspberry pi boards at the time of this post A, B, and Zero. Please don’t attempt to buy the Compute modules. They are out of the general discussion and only are relevant to circuit designers with enough skill levels to integrate into their products.

Originally there were only two models, the model A, the cheaper with less features and the model B, the full-feature one. There are three generations of them already, gen 1, 2, and 3. Major improvements were made between generations, such as doubling memory and using a quad-core in gen 2 or adding wifi/BT in gen 3. When they make smaller improvements to their designs, such as using a faster processor or better Wi-Fi module, they will put a “+” after the model, such as 3B is superseded by 3B+ with faster processor, faster ethernet, more recent Wi-Fi and Bluetooth connectivity, and more power-hungry than ever.

The model B boards are the main stream boards. The model A boards have less memory, single USB port and no Ethernet, and at times not offered to contrast every model B offering. They have always kept the $35 price tag for B and less for A, such as $20 or $25. The intent of model A is to have it run a project that doesn’t need all the USB ports or Ethernet. Say you want to run some slides on a big screen, you don’t really need more than just the processor and sd card. USB ports or Ethernet are optional. You can save money with model A. Last time I visited Minnesota Institute of Art, I took a peak behind their big screens. They were mostly using model B (probably gen 2 or gen 3, not the plus with metal heat sinks)! I couldn’t take a photo or investigate more though. It’s an art museum any. People go there to see art, not raspberry pi?! Anyway, that would be a use case for model A.

Now (in 2015) the raspberry pi folks wanted to challenge themselves to come up with a computer as cheap as $5 (again accessories count as extra), possibly because many other folks started spinning their own boards with similar specs to raspberry pi and many advertised for low prices. So they did it with model Zero, cramming all that raspberry pi goodness on a board only a third the size of a regular pi. Apparently it is not a prequel to gen 1. They slapped the same processor their gen 1 was using and did away with USB or Ethernet, resulting in a much smaller (only on the look) board that they are selling for $5. Is it really that cheap? I’ll never know. They only sell you limited quantities, such as 1 per customer.

Apparently their successfully answered the challenge to themselves and the community was clamoring about it. They designed a sequel, the Zero-W, with a Wi-Fi/BT on board, same as the Wi-F-/BT on their 3B+, for $10 each. This is better than the Zero, since you can’t really interact with the Zero easily. You first need a mini-HDMI to HDMI adapter to bring out the video. Then you need a USB-otg adapter to hook up a keyboard/mouse combo. If you have the Zero, you have no network! You can install nothing or update nothing. You need to get a USB hub and a USB-Ethernet dongle, for another $10 or more. This newer model is again on limited stock. You can buy more than one if you wish to pay a premium of $15 each, or $20 each if you wish to get more than half a dozen. Definitely these Zero models are NOT for any practical projects that need to be deployed at more than a few locations.

Performance

If you wish to use a raspberry pi as a desktop computer, definitely go with 3B+. 1A and 1B are both obsolete, having single-core sub GHz processors. My oldest raspberry pi is a 1B. It’s painfully slow as a general-purpose computer. I was disappointed at it since the original goal of this device is to teach kids computing. The two mounting holes that were haphazardly placed on the board like an afterthought also bothered me. I did a few projects on it. But nowadays it is sitting idling in its case, inside my box of raspberry pi stuff. In 2014, they cleaned up the board and pushed out 1B+. It’s much easier on the look, and to use, since it comes with 4 USB ports and 4 symmetric mounting holes. This footprint has become the standard for the rest of their models A and B boards. The mounting holes and connectors would be at the same location across multiple generations, and they don’t plan to change. Same processor is used on the model Zero series so far, with a single-core processor now overclocked to 1GHz and 512 MB ram. I do use one of them, the one without Wi-Fi, as a data logger. I had to tether all the wires, USB-otg, to a USB hub, then a USB-Ethernet dongle and a sensor on USB. But since it’s sitting in my garage headless (no keyboard or monitor), I don’t need HDMI or keyboards. I could use the Zero W but still have to have the same stuff due to the sensor on a USB port. Gen 2 only has 2B, no 2A. It’s obsolete as well although you can still buy them. They are better in desktop performance already but would require Wi-Fi dongle to connect to home network. I used to tether my 2B to an Ethernet port on my home Wi-Fi router. Now it’s sitting inside another box with a preloaded program that I might run as a demo. 3B gives you pretty decent desktop performance, comparable to a netbook with Atom processors. I got a retropie running on it, emulating old video game consoles. It has no problem running those emulations. The 3B+ is pretty nice. I use it when I need to get some work done on it. I now have a 3A+, which sports the same processor as the 3B+, with half the memory. It runs fine. I just want one for my small collection and in case I need it for something (probably never, other than blogging about it).

Projects

There are some projects that you need the smaller footprints and you don’t mind soldering/desoldering, you go with ZeroW. Performance is low, as space is premium for your project. You can add more USB ports with custom extension boards but the size and price both go up. You can connect a camera to it too. If you rely on adapter wires to get to the USB port for instance, then your project size will likely double due to the difficulty of bending that adapter to fit in place. Trust me, trying to organize cables with “huge” connectors such as Micro-USB is no fun.

If your project is not limited by size as much, but doesn’t benefit from having more than one USB port, such as a big TV slide show/display case/kiosk, and you potentially want to deploy a number of them, go with model A (3A+). You don’t need the added USB port, Ethernet, or double memory for those tasks.

If you want more flexibility for your project, go with the model B (3B+). You won’t be disappointed. There is a chance you can trim down the requirements into 3A+ in case you deploy your project in many locations.

Note: all photo credits to raspberrypi.org or myself 🙂

Filed under Raspberry pi Tagged with

Raspberry Pi 3A+ vs. 3B+ and all those other models

February 3, 2019 1 Comment

This post is intended for those that are considering starting their journeys with raspberry pis or just want to know a bit more about what raspberry pi boards offer which features. If you are a raspberry pi history buff, or raspberry pi advocate (me too but for different reasons than yours), read on as well. Leave me some comments! Not only are specifications of raspberry pi board explained, but also my take on the question “why you want a raspberry pi”. If you wish to know more about the differences between different raspberry pi boards, read my next post.

The raspberry pi folks have released another version of raspberry pi, this time, the 3A+ version. I grabbed one right away when my local electronics store got these boards in stock. While I was browsing accessories (believe me you can never have enough raspberry pi accessories), trying to find the official white/raspberry 2-color case, I overhead a couple of people talking about raspberry pi. They were just reading off the labels and that was already confusing enough for them, 3B, 3A+, 3B+, zero-W etc. I know from their voices that they were not being sarcastic, just genuinely confused what’s what. They walked away before I could strike a conversation with them. I thought, if you are just starting with raspberry pi, then what are all these designations and more importantly which board should you get?

The second question is simple, beginners should get the best model, the 3B+ model with the fastest processor + most memory + most USB ports etc. The prices only differ as little as $10. The cheaper ones are meant for more specific applications.

What you get for a rapsberry pi 3B+ is the following features for $35 (extra for required accessories I’ll talk about later). Every feature has my comments for beginners:

Raspberry Pi 3B+:

  • Quad-core 1.4GHz processor: This is a fairly decent processor not too different from the processor of a netbook around $200 (Broadcom BCM2837B0, Cortex-A53 (ARMv8) if you want the details)
  • 1GB memory: This is not a huge amount of memory but the Linux OS is not a memory hog that windows is. (LPDDR2 SDRAM to be exact)
  • Most recent version of Wi-Fi and Bluetooth: you can connect to your home Wi-Fi in seconds although Bluetooth connection can be a struggle just because Bluetooth itself is such a struggle to use (2.4GHz and 5GHz IEEE 802.11.b/g/n/ac wireless LAN, Bluetooth 4.2, BLE if you want to know)
  • Ethernet: connect to your router with a CAT-6 cable for network if you don’t want to use Wi-Fi (Gigabit Ethernet over USB 2.0 (maximum throughput 300 Mbps) to be exact)
  • Extended 40-pin GPIO header: for tinkering with electronics, such as blinking an LED or controlling motors to your custom robot (most accessories sold for raspberry pi are compatible with this pin layout)
  • Full-size HDMI: this is nice so no adapters needed, just straight HDMI cable
  • 4 USB 2.0 ports: minimally you need a keyboard-mouse combo dongle, maybe a flash drive and other devices such as a gamepad, or Arduino etc.
  • CSI camera port for connecting a Raspberry Pi camera: this connects to a camera with special interface for future projects. The camera, which is a $25 separate official accessory, comes with no enclosure or mount. You need an enclosure or separate stand to make it useful. It is also a nice thing since you can make your own design where the camera goes on your own enclosure.
  • DSI display port for connecting a Raspberry Pi touchscreen display: this connects to an official display with special interface for future projects. The display is rather expensive at $60 and needs some assembly. Again it comes without a stand, just the display and an exposed driver board. You need to buy a stand or make your own so the pro of flexibility and con of not a complete solution.
  • 4-pole stereo output and composite video port: probably not useful for most people and projects unless you want your old TVs for display.
  • Micro SD port for loading your operating system and storing data: you need a MicroSD card at least 16GB since raspberry pi has no storage, costing you extra $10.
  • 5V/2.5A DC power input: this is just a MicroUSB socket. You need a USB-to-Micro USB cable (aka Android charging cable) and a phone charger. The required power is rather high so you can’t use your old Android phone charger. I have a couple of generic AmazonBasics chargers with 2A current. Make sure you get a short fast charging cable with thick conductors. Those generic Micro-USB cables have very thin wires that drop too much voltage on themselves so what they deliver to your raspberry pi is often insufficient, causing all sorts of problems.
  • Power-over-Ethernet (PoE) support (requires separate PoE HAT): probably not a beginner’s feature. With the official accessory (a HAT), you can deliver both internet and power through a single CAT-6 cable to your raspberry pi, de-cluttering cables by a great degree. The accessory is “expensive” considering it is $20 extra dollars just to get rid off the Micro-USB charging cable. That’s not the end of it. The other side of the CAT-6 cable, the router, needs to support PoE and has PoE injector. The HAT also covers all your 40-pin headers, so no more tinkering with electronics.

So let’s see, you get a netbook-level computer for $35. Add the following accessories that you may already have:

  • $10 for 16GB+ MicroSD card
  • $10 for USB charger
  • $5 for charging cable
  • $20 for the cheapest keyboard and mouse combo
  • $5 cheap HDMI cable
  • $5 cheap case (official version is $9)

This isn’t everything. If you don’t have a monitor, that’s extra $$$. So the minimal equipment cost is $35. A realistic cost is maybe $75. If you want to set up everything from scratch, it’s $100 plus whatever the cost of a monitor. By this time, you should realize that buying a raspberry pi is not going to save you much money from buying a new PC. A PC laptop with the same if not better spec can be bought for less than $200. You can take said laptop anywhere you want and use it for hours between charges. Raspberry pi can’t do any of that! There are kits you can buy and assemble yourself to make a raspberry pi into a clumsy laptop if you wish to dish out $300+.

So apparently a raspberry pi is NOT a cheap PC replacement! You will NOT save money buy purchasing a raspberry pi! Should you still get it? If you’re deciding between a cheap computer for school and rent, you should just go with a cheap computer. If you have $100 extra money you saved up for “new cool tech things”, you should buy a raspberry pi. That’s right! Raspberry pi is a HOBBY! I don’t remember having hobbies when I was poor. That’s OK. Unless you’re really struggling to support a family, you can find money for this one hobby. Raspberry pi is not an expensive hobby. If you are interested in technology, this hobby will pay you back in many ways that exceed your already-moderate and low-risk investment. Delay that “smart phone” upgrade! All of a sudden, you have extra money! Make yourself sandwiches or some real food, use a few coupons at supermarkets, bring fruit and soda to work/school, come early and find free on-street parking.

Anyway, the real “specs” that I would like to say, are not measured in Gigabytes or GigaHertz. They are measured in more qualitative and subtle things:

  • Take a journey in popular computing technology, with millions of stops depending on what your interests are
  • Become a maker and have computers, sensors, motors, the internet do things for you, not just what you can buy from the stores
  • Be part of an extremely creative global maker community with people from all walks of life with all backgrounds that all share the love for computers and innovation

 

Filed under Raspberry pi Tagged with

Open FTDI USB-serial UART port by ID

July 3, 2018 Leave a comment

In this post, I will explain how to open serial port to your Arduino or SDI-12 USB adapter by its unique ID so you always open the correct port even when there are multiple such devices on your computer or raspberry pi.

For Arduino and SDI-12 USB adapter users, I have a nice trick to help you manage multiple Arduinos or SDI-12 USB adapters on the same computer or raspberry pi. On raspberry pi, as on a typical linux system, your device shows up as a serial port, such as /dev/ttyUSB0. This serial port designation is usually bound by the order that the device is discovered at boot time, which may not be the same even if you keep your adapter plugged into the same USB port. This means if you have more than one device on your raspberry pi, you may open the wrong port at times, which should be a big issue. To prevent your program from opening the wrong port, you need a unique ID for each device. Luckily FTDI chips already come with unique IDs. We just have to find those IDs and possibly change them into more meaningful things for us to remember. Assume for the moment you are making a data logger for your test fields. There is one field that can be called “NORTH”. The following steps will help you change the ID of the FTDI chip on your device so you can later open its port by that ID, instead of a port name. Here is a list of which devices are using FTDI’s chips that have the reprogrammable ID feature:

  • Liudr SDI-12 USB adapter (all types)
  • Sparkfun Redboard
  • Certain Arduino clone boards
  • Lots of other devices such as GPS etc.

Here is the FT_PROG tool FTDI provides. It’s windows only but I’m sure you can find a windows machine to run it. I’ve not tested it in a virtual machine whose host is linux or macos. I’ll do that when I have more time. If you are unsure whether your device has an FTDI chip, a quick scan using the program will tell you.

http://www.ftdichip.com/Support/Utilities.htm#FT_PROG

First, press the scan icon (magnifying glass). If you have a device with FTDI chip, it will show up. See the screen grab below:

So I have an FT232R chip with a chip serial number “A106DHE5”. I can open port with this serial number but I’d rather change it to “NORTH”. Click on the “SerialNumber” from the left side.

Uncheck the “auto generate serial no” so you can edit the serial number to “NORTH”. You have up to 16 characters to name the adapter. Once done, press flash icon (thunder bolt).

Now that you have programmed your chip, you can read the information back using “scan” again to verify that the ID has changed:

Now that you have this nice ID, let’s open port by this ID.

Here is a small complication. On linux, the chip ID is returned, such as “NORTH”. On window, the port ID is returned, such as “NORTHA“. The addition of the “A” indicates the port “A” on chip “NORTH”. This is because some FTDI chips have two serial ports. The port IDs will be “NORTHA” and “NORTHB“. Even for FTDI chips that have only one port, such as for our case, the “A” is still there. So I recommend comparing chip ID instead of port ID. If you only work on linux/rpi systems, this doesn’t seem to concern you. But if you wish to make your code platform independent, i.e. running on windows without an incident, you will only extract whichever ID you receive with the stored chip ID, up to the length of the stored chip ID. Note: in the platform-independent code, you can’t slice a port’s serial_number with your stored ID because some internal ports don’t have serial numbers thus returns empty that will throw an error when you try to slice an empty array.

The following is a snippet that works ONLY on linux/rpi systems:



[sourcecode language=”python” wraplines=”false” collapse=”false”]

import serial.tools.list_ports  # For listing available serial ports

import serial  # For serial communication


my_ID=’A817EQLG’

port_device=”

# List ports for user to select

a = serial.tools.list_ports.comports()

print(‘\nDetected the following serial ports:’)

for w in a:

    print(‘Port:%s\tID#:=%s’ % (w.device, w.serial_number))

    if (w.serial_number==my_ID): # Match ID with the correct port

        port_device=w.device # Store the device name to later open port with.

if len(port_device)!=0:

    print(‘\r\n%s is the correct port.’ %(port_device))

else:

    print("Port with ID: %s is not found!" %(my_ID))

[/sourcecode]


The following is a snippet that works on ALL OS:


[sourcecode language=”python” wraplines=”false” collapse=”false”]

import serial.tools.list_ports  # For listing available serial ports

import serial  # For serial communication


my_ID=’A817EQLG’

port_device=”

# List ports for user to select

a = serial.tools.list_ports.comports()

print(‘\nDetected the following serial ports:’)

for w in a:

    print(‘Port:%s\tID#:=%s’ % (w.device, w.serial_number))

    if (w.serial_number.__str__()[:len(my_ID)]==my_ID): # Match ID with the correct port

        port_device=w.device # Store the device name to later open port with.

if len(port_device)!=0:

    print(‘\r\n%s is the correct port.’ %(port_device))

else:

    print("Port with ID: %s is not found!" %(my_ID))

[/sourcecode]


Your choice, simplicity of code or cross-platform compatibility.


Here is the cross-platform code’s result on my windows machine:


Detected the following serial ports:
Port:COM23	ID#:=A817EQLGA
Port:COM3	ID#:=None


COM23 is the correct port.



As you can see, there is an added “A” at the end of the ID reported by windows, which the python code ignored to produce a match.


I’ve also attempted to do this using Processing 3.0. Unfortunately, the Serial.getProperties() function that should return similar information returns blank (possibly not implemented on windows and yet to be tested on linux). If you have tested Processing method with success, please reply below with your results. I’ll add your comment to the post.


Here is the code I used in Processing 3.0:


[sourcecode language=”java” wraplines=”false” collapse=”false”]

import processing.serial.*;

import java.util.Map;


void setup() {

  String[] ports = Serial.list();


  for (int i=0; i < ports.length; i++) {

    Map<String, String> props = Serial.getProperties(ports[i]);

    print(ports[i]+": ");

    println(props);

  }

}

[/sourcecode]


Results:


COM3: {}
COM23: {}
:(



Closing note: even if you work on Windows that assigns unique COM port number to your arduino or adapters, the assignment relies entirely on the currently available port numbers. If you develop your project on one windows PC and deploy on another windows PC, you WILL get different COM port numbers. On a Mac, the ID is embedded on the port name such as /dev/ttl.usbserial-A103RU9T so you are better off. But, will you be willing to shell out the money to get a mac and have it sit somewhere to collect data just because of this feature? If you are a linux wiz, you can bind names with serial numbers using some scripts. That’s beyond the scope of our general discussion, which assumes minimal experience with linux administration.




				


				

					

						Filed under Arduino, data logging, Python, Raspberry pi
											

				


			


		
			


				


					
Can’t upgrade pyserial in latest raspbian distribution?


					

						

							July 18, 2017
																						Leave a comment
																				

					


					
This is just for your information if you are a Raspberry Pi user and playing with Python code from my blog. If you are trying to use the latest distro of raspbian with pyserial for some serial port project, you may have come across this issue that regardless how you upgrade pyserial using pip3, your python3 will always call up the old pyserial 2.6 that came with the distribution. I am a bit disappointed that the foundation has included such an old version of pyserial, couldn’t they just try a pyserial 3.0 instead? My solution was to remove the python3-serial module using apt-get and then install pyserial 3.3 using pip3.


[code language=”bash”]

sudo apt-get remove python3-serial

sudo pip3 install pyserial

[/code]


Hope this helps.




				


				

					

						Filed under Announcements, Python, Raspberry pi
						Tagged with , 					

				


			


		
			


				


					
Saving data to sparkfun server


					

						

							June 27, 2016
																						Leave a comment
																				

					


					
If you’ve been following my posts, I discussed how to use my SDI-12 USB adapter and raspberry pi (or PC) to log data from SDI-12 sensors. Now that you have your data, you can also send them to sparkfun’s data server for storage and later retrieval. In the following video, I explained what you can do with this feature included in my data logger python script. I also constructed a sample webpage to display data saved on the server. Lots of customization can be made based on this sample webpage.


 






				


				

					

						Filed under Raspberry pi, SDI-12, Sensors
											

				


			


		
			


				


					
30-day temperature data


					

						

							June 5, 2016
																						Leave a comment
																				

					


					
Since I was going away for a month, I decided, before I left, to set my SDI-12 data logger to run a 30-day data collection routine, just to see how robust the software is. Here is the result:


30-day temperature


The temperature sensor was in my office so its variation was small. The steep slope towards the end was because I opened the window after I got back and kept the window open for a whole day. You may notice the lack of variation of temperature between 5/10 and 5/13 but that WAS what happened. History data from wunderground.com showed less-than-average variation of outdoor temperature in my area during the same period:


wundergroundSo my conclusion was that my python data logger code running on raspberry pi was robust enough for at least 30 days so was the SDI-12 USB adapter. If this were done outdoors, then the raspberry pi and the SID-12 USB adapter both need protection from the element. Also a solar panel and battery will be needed.


Parts and components:


Raspberry pi 2 B (or 3 B)


SDI-12 USB adapter (Free python data logger code)


Decagon 5TM soil sensor


 




				


				

					

						Filed under Python, Raspberry pi, SDI-12, Sensors
											

				


			


		
			


				


					
Soil data logger telemetry


					

						

							April 24, 2016
																						2 Comments
																				

					


					
I have finally found time to build a simple website for my soil data logger with telemetry. The system works as the following:


    

  1. The data logger consists of a raspberry pi and my SDI-12 USB adapter with a Decagon 5TM soil sensor
    2. 

  2. The data logger runs the open-source datalogger code I wrote in Python to first get parameters from the user (COM port, SDI-12 address, delay etc.), and then collect data, save to a local .CSV file, and then send the same data to sparkfun’s phant server.
    3. 

  3. I constructed a web interface to plot the data using Google Charts and download .CSV version from sparkfun’s phant server.
    4. 



Here is a screen shot:


soil logger webpage


I’ve uploaded the webpage to a server with a link below. The sensor is apparently NOT buried in soil so I can easily take the setup and set it up in different places to test its stability.


Link to the website: Link




				


				

					

						Filed under Announcements, Python, Raspberry pi, SDI-12, Sensors
						Tagged with , , , 					

				


			


		
			


				


					
Back up and clone raspberry pi


					

						

							March 25, 2016
																						32 Comments
																				

					


					
In this post, I will explain how to back up and restore or clone a raspberry pi. I am assuming that you have installed Raspbian or Ubuntu mate on your raspberry pi. Since a raspberry pi runs its OS entirely on an sd card, it is easy to clone your system so you can pass along to a friend or create your special distro for others to use. For example, a company that makes compact optical spectrometer, Ocean Optics, has created a spectrometer program that runs on raspberry pi and has its own web interface. You just have to download their raspberry pi image and put it on your card. No need to do multiple build from source and apt-get etc. to get what you need.


If you are already a Linux guru, this post is NOT for you. I am assuming that you don’t have a Linux machine other than raspberry pi or you are still learning Linux.


The “easy” way:


Raspberry pi foundation has raw images of their Raspbian and NOOB on their website. On a windows system, all you need is win32 disk imager. It reads in the image and writes it on an SD card. You can save the image it reads and keep it as a backup or use it to clone a system.


The catch:


Since win32 disk imager only reads and writes in raw format, it actually doesn’t know what it is reading/writing. Say if you want to clone your raspberry pi 2B running Raspbian Jessie on an 8GB Kingston microSD card onto an 8GB SanDisk microSD card, you simply can’t. Although both cards claim to be 8GB, the SanDisk card is about 20MB smaller than the Kingston. Not understanding what it reads and writes, win32 disk imager is unable to shrink even one byte of what it reads (the whole 8GB from Kingston) to try to fit it onto a smaller card. This has pushed people into buying larger cards, such as 16GB. Then when they try to clone a system again, they run into the same issue unless they have bought a few identical SD cards as clone targets. Even that has issues (read the end of the post for details). So they have to step up to 32GB!!!


The solution:


We need a disk/partition reader and writer that understands what it is processing and is able to resize the partitions so they fit. If you are not hosting a lot of large files, you should be OK with an 8GB card or more than OK with a 16GB card. I’ve looked around for quite some time and found my solution: Paragon backup and Recovery 14 Free edition (Home edition for $39,99 has more features not useful for us). It is only for windows so those mac users will either need to become Linux gurus or shell out a small amount of money to get a win 10 netbook.


What the program does is that it is able to back up a whole disk or SD card and restore it on disk or SD card of different size. The reason is it understands most common file systems such as FAT, NTFS, HFS, and EXT. Besides, it also automatically compresses the backup so it doesn’t take more space than it should on your PC.


How to do it:


First you make a backup of your raspberry pi card using back up to VD. It is a virtual hard drive. This backs up both BOOT partition (FAT partition) and your Linux partition (ext4).


backup_to _vd


Next, insert your new SD card (you need to quit the program and restart it). Select restore from VD. It will ask you whether to resize partition and you can choose yes. This way you can even squeeze your clone from a larger card (32GB) to a smaller card (8GB) if your Linux partition has a lot of space.


Another neat trick:


What else you can do is to resize BOOT partition. Although BOOT partition has not much use for most of us, it IS the only partition that windows will recognize. If you are using your pi as a data logger or need to copy files from and to it (without the hassle of SSH and network setting), then the best way to get data is if it is written to the BOOT partition. You turn off raspberry pi and put its SD card on your windows machine. Immediately you have access to BOOT partition. You can copy your data out, you can change some config files of your data logger etc. It is useful to have a large BOOT partition. To do this, you will have to restore one partition at a time. Restore BOOT first. You will be prompted to select the size of the restored partition. Make your selection, say 500MB. Then when the restore is over, use the rest of the space on card to restore your Linux partition (make sure you leave enough space for Linux partition).


Why buying identical cards may not solve clone/restore issue?


Well, say you purchased 10 Kingston 8GB microSD cards, they are all the same size, correct? Wrong! I learned it the hard way. I had a friend that sent me an image he saved from such a card. I have identical cards. But, when I was telling win32 disk imager to write the image on my card, it complained: the target card doesn’t have enough space! Guess what, the target card is one sector less than the image file has?! But how can this happen? It’s simple now that I got my answer: some sectors on the SD card may have become bad either at factory or post purchase. The SD card controller (inside the card there is a controller) has disabled those sectors, making not all “identical” cards identical.


 




				


				

					

						Filed under Raspberry pi
											

				


			


		
		

			
← Older posts

			

		


	












		
		

		
				
		
		
	
				







		
				
			
	
%d