Cloudant DBaaS 101

I have been playing with http://www.cloudant.com DBaaS and I am very impressed. If you want a NoSQL scalable database goto https://cloudant.com/sign-up/ and create free account. They have reasonable rates once you go beyond the free tier but you will never get a bill if you use less than $50 a month (approx 1,600,000 light reads or 300,000 heavy writes) or 50GB. Cloudant can handle billions of transactions a day, replicate data across data centres, secure/encrypt your data etc

The hardest part is understanding how to prepare HTTP requests to the server (and formatting/reading the JSON payloads) but that is way simple than dealing with the hell of SQL and HTTP forms, sessions and SQL injection stuff. When you signup choose your name (e.g “testuser”) this will become the URL to your database (“https://testuser.cloudant.com”). You can also choose where your database is located (US, London, Singapore etc) but don’t worry, this can be changed later. By default all traffic to/from cloudant is via https (at no cost) unless you opt out.

If you are unsure just goto https://cloudant.com/sign-up/ and wait 5 min and a popup live chat window will appear and ask away. You can also follow @cloudant on twitter, join the community on IRC or email support@cloudant.com.

My blog post is certainly not going to teach your the right terminology (for that go here https://cloudant.com/for-developers/ and https://docs.cloudant.com) but in a few minutes you can easily create a table, write new records, query and modify records in minutes.

How to create a table

Once you have created an account you can create tables (database) via the Cloudant url (https://{yourusername}.cloudant.com/dashboard.html) or remotely via a number of ways (e.g a Python Script). You do not need to use Python, you could use .NET, HTTP code from an Arduino or PHP code on a website. As long as you can send a http request with authorisation (login details) and attach/decode JSON you can talk to/from a Cloudant DBaaS.

import requests
import json
auth = ('yourusername', 'yourpassword')
post_url = "https://yourusername.cloudant.com/testtable".format(auth[0])
r = requests.put(post_url, auth=auth)
print json.dumps(r.json(), indent=1)

This will create table called “testtable”

If you do something wrong cloudant will let you know via the API and HTTP response code https://docs.cloudant.com/api/basics.html. Always check the response codes.

How to save a new record into a table

Obviously you need to decide on the table names and fields in your database table but here is a quick way to throw a record at the database above with python.

import requests
import json
doc = {'username':'kerrie','high_score':550, 'level':3}
auth = ('yourusername', 'yourpassword')
headers = {'Content-type': 'application/json'}
post_url = "https://yourusername.cloudant.com/testtable/kerrie".format(auth[0])
r = requests.put(post_url, auth=auth, headers=headers, data=json.dumps(doc))
print json.dumps(r.json(), indent=1)

You can see that we are creating a JSON array (containing some key pair values and passing that to a URL via HTTP packet data).

How to query a record

Simply query the URL (http://yourusername.cloudant.com/table/record

and process the JSON returned.

import requests
import json
auth = ('yourusername', 'yourpassword')
get_url = "https://yourusername.cloudant.com/testtable/kerrie".format(auth[0])
r = requests.get(get_url, auth=auth)
print json.dumps(r.json(), indent=1)

There is a whole topic on querying by ID’s and versioning that is not covered here.

How to query all record

You can also return all records in a table by querying the URL: https://yourusername.cloudant.com/testtable/_all_docs

Results would be returned something like this:

{"total_rows":4,"offset":0,"rows":[
{"id":"jane","key":"jane","value":{"rev":"5-t8ca55e2c81d38895ba8eff46a53c5db"}},
{"id":"kerrie","key":"kerrie","value":{"rev":"4-5e21e7e140af02afdc2f37b7538bcc34"}},
{"id":"marc","key":"marc","value":{"rev":"5-ha8a12fc9829baa09c5940edcr6fdf9f"}},
{"id":"simon","key":"simon","value":{"rev":"6-a9e10252931e4df63aa5ce055c56aac9"}}
]}

There is also syntax for maximum items returned and pagination that you can read up on.

Now that you understand the basics you can plan how you want to talk to your database, it could be via .NET, PHP, Perl, Corona SDK or via Python. Do check out the official create/read/update and delete examples and read the getting started guide.

In the next post I will add some CoronaSDK to Cloudant tips.

Edit 2/Jan/2015: Cloudant have increased the freebies 10 fold (I have updated the prices above).

Arduino GPS Code for GPS/GPRS/GSM Module V3.0 (SKU:TEL0051) loaded

Arduino GPS Code for GPS/GPRS/GSM Module V3.0 (SKU:TEL0051) loaded.

20140727-031035-11435603.jpg

http://www.dfrobot.com/wiki/index.php/GPS/GPRS/GSM_Module_V3.0_%28SKU:TEL0051%29#How_to_drive_the_GPS_Mode_via_USB_port

Quite a bit of research to get the boards code, configure the arduino IDE and latest GitHub code.

Now I have to go outside to get a GPS signal.

20140727-031152-11512745.jpg

Update: Was able to send a text message from the Arduino/gsm module Telstra pre paid sim via power pack alone. It turns out we need to redirect pins 3/4/5 to 11/12/13 :)

20140727-162112-58872675.jpg

20140727-162734-59254900.jpg

Checking (and verifying) the internet on a raspberry pi with a bash script

This is how I check the internet with a bash script on a raspberry pi. It verifies the internet by downloading a file from a webserver (e.g http://www.your-domain/pi/test/simple_text_file.txt
(change this to a file of your choice)).

Script: CheckInternet.sh

    #!/bin/bash
    printf "BASH Version: $BASH_VERSION\n"

    printf "Testing Internet Connection (http://www.your-domain.com):\n\n"
    # ping -q -t 10 -w1 -c1 www.your-domain.com
    wget -q --tries=2 --timeout=20 http://www.your-domain.com
    if [[ $? -eq 0 ]]; then
    printf "Internet appears online.\n\n"
    TMP_FILE="./simple_text_file.txt"
    if [ -f $TMP_FILE ]
    then
    echo "Deleting old temp files.."
    sudo rm ./simple_text_file.txt
    fi
    printf "Downloading file from the internet.\n"
    sudo wget http://www.your-domain/pi/test/simple_text_file.txt
    if [ -f $TMP_FILE ]
    then
    printf "File downloaded ok.\n"
    printf "File Contents:\n"
    echo "----------------------------------------------------------"
    printf "\n"
    cat $TMP_FILE
    printf "\n\n"
    echo "----------------------------------------------------------"
    else
    printf "File failed to download.\n"
    fi
    else
    printf "Internet appears offline, unable to download files."
    fi

    printf "Tiding up..\n\n"
    sudo rm ./simple_text_file.txt

    printf "Goodbye\n\n"

To call this script from the command line do this.

    sudo bash CheckInternet.sh

Alternatively you can schedule emails by adding this to your “/etc/crontab” file


    # run every 15 minutes
    */15 * * * * root bash /usr/bin/simon/CheckInternet.sh

    # run every 5 minutes
    */5 * * * * root bash /usr/bin/simon/CheckInternet.sh

Or you can run this on pi start-up by adding the python call to your “/etc/rc.local” file.

    bash /usr/bin/simon/CheckInternet.sh

Make sure you make you file executable.

Send Email from Raspberry Pi

This is how I am sending emails (boot and 15 min runtime notifications) from my Pi

Python Script: /usr/bin/simon/SendEmail.py

You will need to type in your mail server Username/Password/IP/Port to get this to work.


    #!/usr/bin/python
    import smtplib

    EMAIL_USER = 'your-email@your-domain.com'
    EMAIL_PASS = 'your_email_password_goes_here'
    SMTP_SERVER = '192.0.0.1'
    SMTP_PORT = 587

    def send_email(recipient, subject, msg):
    smtpserver = smtplib.SMTP(SMTP_SERVER, SMTP_PORT)
    smtpserver.set_debuglevel(0)
    smtpserver.ehlo()
    smtpserver.starttls()
    smtpserver.login(EMAIL_USER, EMAIL_PASS)
    header = 'To:' + recipient + '\n' + 'From:' + EMAIL_USER + '\n' + 'Subject:' + subject + '\n'
    header = header + '\n' + msg + '\n\n'
    msg = header + '\n' + msg + '\n\n'
    smtpserver.sendmail(EMAIL_USER, recipient, msg)
    smtpserver.close()

    send_email('recipient-name@gmail.com','Subject Goes Here', 'Email contents goes here.')

To call this script from the command line do this.

    sudo python SendEmail.py

Alternatively you can schedule emails by adding this to your “/etc/crontab” file


    # run every 15 minutes
    */15 * * * * root python /usr/bin/simon/SendEmail.py

    # run every 5 minutes
    */5 * * * * root python /usr/bin/simon/SendEmail.py

Or you can run this on pi start-up by adding the python call to your “/etc/rc.local” file.

    python /usr/bin/simon/SendEmail.py

Setting up eth0 and wlan0 on a Raspberry PI RevB (no drivers)

Recently I setup a raspberry pi to talk to the internet via ethernet here. I asked myself how do you configure Ethernet and wireless on the radpberry pi?

Today I added a wireless card and set it up so it can talk to the network too.

Step 1: Wireless Dongle.

I purchased 1x EDUP Model EP-N8537 Wireless LAN 150Mbps USB Adapter from eBay for $7.95 ( , item).

wlan0b

Step 2: Configuration.

I plugged my new Wireless card into Windows and ran the following command to get the MAC Address for the new card. Write the Mac Address down.

    ipconfig /all

I then connected to my ADSL router and went to the Static IP section under LAN/DHCP.

I added a static IP reservation for this mac address to my router and gave it an IP that I was happy with.

I then booted the Raspberry Pi (don’t connect the Wireless card yet). I typed the following command to open the network interfaces file.

    sudo nano /etc/network/interfaces

I deleted the contend of my interfaces file and typed this in.


    auto lo
    face io inet loopback

    iface eth0 inet static
    address 192.168.1.200
    netmask 255.255.255.0
    network 192.168.1.0
    broadcast 192.168.1.255
    gateway 192.168.1.1

    auto wlan0
    allow-hotplug wlan0
    iface wlan0 inet static
    address 192.168.1.204
    netmask 255.255.255.0
    network 192.168.1.0
    broadcast 192.168.1.255
    gateway 192.168.1.1
    wpa-ssid yourwirelessnetworknamehere
    wpa-psk yourwirelesspasswordhere

If you are unsure of what to type above go and check out the hundreds of awesome and free security/networking podcasts here https://www.grc.com/securitynow.htm.

Step 3:Reboot.

I ran a safe shutdown, inserted the WLAN adapter and rebooted.

Now I can use the ethernet or WIFI.

wlan0a

No drivers or non fee DEB packages were installed to get this NIC working.

Bonus tip: I am using a Ethernet over Power adapter connected to a wireless extender to allow my arduino/raspberry pi to connect to my router.

Arduino/Pi -> Gigabit Switch -> EoP A -> 240v (50m) -> EoP B -> 100MB Switch -> WiFi (30m) Extender A -> WiFI -> WiFi Extender B -> Router -> Internet :)