I just got back from my vacation in Shenzhen yesterday and was going to relax a bit until Monday when I’d start working on some new designs. When I was reading the New York Times this morning, a particular article caught my attention. Hiroko Tabuchi wrote an interesting article about the worsening crisis going on at the Fukushima Dai-Ichi nuclear power plant.

The radioactive waste water is leaking into the ocean and the leaks seem to be getting larger. There are nearly 1000 tanks at the plant to store the radiated water which are prone to leaks. Also, there are no water level gauges in the tanks and only two men patrol the tanks every day to check for leaks. What caught my attention in the article was when it mentioned TEPCO had no reliable way to check the storage tanks for leaks. That was actually one of those “hmmm….really” moments for me.

The problem of detecting water leakage in the tanks can actually be restated as a need for continuous monitoring of water levels, a common problem. In fact, I ran a workshop in Dharamsala, India in the Himalayas earlier this year to teach locals to implement automated water level monitoring in storage tanks that provided drinking water to Tibetan Buddhist monks and students.

2013-04-12 DSC03305

There are a lot of different solutions for water level monitoring, but in this situation, there are a couple of constraints. The structures need to be retrofitted without disturbing the water, doing anything to come in contact with the radiated water is probably undesirable, and any access to the storage tanks are probably at the top of the tanks which are likely a few stories tall. This means that access is extremely inconvenient.

This is actually an ideal case study for a wireless sensor network. A remote device consisting of a battery powered, water level sensor and a wireless transmitter would be placed on top of each storage tank. The remote devices would communicate with an aggregator device connected to the internet and uploading data to a server periodically. Finally, the problem is then reduced to parsing web data to visualize the tank levels and possibly color code or send alarms when the water levels in any particular tank start to drop.

Sure it’s all fine and dandy to talk about this theoretically, but let’s go one step further and implement a mock system to continuously monitor water levels in a tank, transmit them wirelessly, and upload the data to the internet.

First, it’s important to take care of the water level sensing part. Based on the limitations I mentioned previously, one of the best ways to handle non-contact water level sensing is to use a sonar based sensor. This type of sensor emits an ultrasonic audio chirp and then waits for the echo. The distance can be calculated based on the time it takes for the echo to arrive. In this case, I’ll be using an industrial grade, waterproof sonar sensor from MaxSonar: the MB7389 . It’s waterproof, pre-calibrated, and has a range of 5m. There are also 10m versions available but I just happen to have the MB7389 on-hand. It outputs analog, serial, and pulse width data, but in this case, I’ll be using the analog output. It’s easier to deal with and at this point, we mostly care about prototyping quickly to check feasibility.

20130324-DSC01465-2

The sensor’s analog output will be fed into the analog input of a 900 MHz freakduino which is an Arduino-compatible board that I make with an integrated 900 MHz transmitter. I’m choosing to use 900 MHz in this case because it has longer range and lower attenuation through free space. By controlling the transmitter power and antenna type, it’s possible to get ranges up to a few kilometers using a stock board.

  2013-08-24 DSC07320

The software side is hopefully pretty straightforward. In the setup routine, we’ll be initializing the analog input pin, initializing the chibi wireless stack, and then setting the channel and address of the device. Since this would be used in Japan, I’m setting the radio to channel 8, the equivalent of 920 MHz which falls in the license free band in Japan. Each device also needs to have a different address with 64k addresses to choose from. In the loop section, we wait for a period of 30 seconds to elapse, read the sensor, convert it to a value, and then transmit it to the receiver. The water level sensor transmitter code can be downloaded here.

This takes us to the receiver side. For this, I’m using the Arashi 900 MHz Ethernet Gateway, another board I made specifically for wireless sensor networks. For the internet data aggregator service, I’m using Xively (formerly Cosm, formerly Pachube). They put out a handy little Arduino library which works great for using their service. You first need to sign up for a free developer account on Xively. Once that’s done, you’ll need to create a data feed page. The data feed page will show the API Key and the feed ID and you can basically just plug both of those values into the source code to set up the Xively interface. FYI, I've never gone beyond the free developer account and I have no connection to Xively.

2013-08-24 DSC07338

Once that’s all taken care of, we need to implement the meat of the code. Most of the setup code is boilerplate code like setting up the serial port and chibi wireless stack. The Xively setup code can be a bit intimidating but you can pretty much just copy it verbatim for each use. In the loop section, we just wait until we receive data wirelessly. When we receive data, I'm doing some minor formatting of the data to convert it to floating point and then just take that data and upload it to Xively. Here's the water level sensor receiver code for the Arashi gateway board.

That takes us past the feasibility check so now we know that it’s possible to implement an automated, continuous monitoring solution to check the water levels in the storage tanks. If you know what you’re doing, getting here takes the better part of a few hours. This actually makes me a little bit sad since two and a half years have passed since those radiated water storage tanks have existed and they still just have two poor guys climbing each tower to check the water levels every day.

2013-08-24 DSC07322 2013-08-24 screens1

Beyond the feasibility check, we now need to fill in the details and harden the design. As I first mentioned, the remote nodes on top of the water towers will be battery powered. This puts some very hard constraints on the design. For wireless nodes, power is everything. We’re going to need to put the system to sleep most of the time so that it will consume as little power as possible. Then, we’ll wake up, take a measurement, transmit it, and then go back to sleep as quickly as possible. To do this, we’ll need to add a switch to power on and off the sensor so that it doesn’t suck down power while the system is sleeping. The software will also need to be written to power the system down in the proper sequence and we’ll have to figure out a method to periodically wake up for the transmission.

Power management is almost an art form and engineers go through crazy contortions to squeeze out every last microamp of current from a design. In this case, we’re going to make a lot of modifications to the software and some to the hardware to reduce the system power consumption. To start off, the main modification to the hardware is to add a power switch in the form of a PNP transistor tied to an I/O on the MCU. I decided to mill a board to accommodate this. It’s really overkill, but it makes things so much prettier and cleaner. On the software side, we initialize the power switch as an output and then enable power to the sensor.    

2013-08-24 screens6-schematic 2013-08-24 DSC07332 2013-08-24 DSC07330

For the wake mechanism, I’m using the watchdog timer interrupt inside the MCU. Normally, I’d use the watchdog timer to reset the system in case it hangs somewhere, but it can also be used to wake the system from a deep sleep mode.

The loop function changes drastically too. Now, rather than loop continously, the MCU will only go through one loop iteration and then go to sleep. Basically, the MCU will wake up, do whatever is in the loop once, go back to sleep again, and wait for the watchdog interrupt to wake it up. The powerdown sequencing consists of sleeping the radio first, and then shutting down different parts of the MCU. The UART and ADC consume a lot of power so those get shut down. Also, the I/O are all forced into input mode so nothing can leech power from them. On wakeup, the reverse happens. After all this is implemented, the system now consumes approximately 330 uA at 2.4V in sleep mode and 82 mA in full active mode. If we sleep most of the time and just transmit data every 10 minutes, the systems can theoretically last over 6 months on a pair of AA NiMH batteries. Not too shabby. You can download the modified transmitter code here .

2013-08-24 DSC07327 2013-08-24 DSC07328

To finish things off, I went out and purchased a trash can to use as a mock water storage tank and then modified it to install the water level sensor.

2013-08-24 DSC07325 2013-08-24 DSC07323 2013-08-24 DSC07324

 I tested the sensor on the empty can. It read 460 mm to the sensor with nothing inside which corresponds to what I saw with my tape measure. I then added approximately 140 mm of water and then tested the sensor again. It now read approximately 320 mm to the surface of the water which also corresponds to both the math and what I measured with the tape measure.

2013-08-24 DSC07336 2013-08-24 screens2-in can

2013-08-24 DSC07335 2013-08-24 screen3-in can boxes

This mock setup is similar to the opening at the top of most water storage tanks. If the storage tanks at Fukushima Dai-Ichi don't have an opening at the top, then it still is feasible to cut or drill a small opening to fit a sensor into. Once there's access, it's possible to automate the monitoring of the tank. Speaking of which, you can view the Xively feed for the mock storage tank in real time here. Note: the feed may be up or down since I'm still doing development on it.  

2013-08-24 screens7-xively

I didn’t want to get too complicated since this was just a spur of the moment prototype I threw together but there are various things I’d do if I wanted to take things further. I’d definitely add a solar input and battery charge circuitry to it so that it could have an indefinite lifetime. It’s already possible to monitor the battery voltage on the board so I’d also transmit the battery status wirelessly. That way, it’s possible to see what the battery charge level is on all devices. Finally, I’d add an external watchdog reset to improve the reliability of the system. At this point, it’s robust enough to deploy and would probably be a much better solution to the storage tank leakage problem than anything TEPCO has currently.

So there you have it. This is my proposal for a solution to the Fukushima Dai-Ichi storage tank leakage problem in tutorial form. It’s both open source software, hardware, uses low cost components, automated, and is much better than anything I’m seeing them use.

Updated 2013-08-24: There has been some discussion on rad-hard requirements for this design. If this is the case, one solution would be to put the electronics in a lead-lined metal enclosure with only the antenna protruding. High radiation fields shouldn't affect communication signals in much the same way as strong light doesn't affect communication signals. I'm also the designer of the radiation monitoring equipment used by Safecast which has been used numerous times in the exclusion zone. There have been no instances of bit-flipping due to radiation that I know of and all the data collected there has come out intact.

 

Updated 2013-09-12: Took the system offline after three weeks. It had a good run, and I'll be repurposing the equipment for some agricultural monitoring projects :)