Wi-Fi connecting the Canon 1DX on the cheap

I've had the Canon 1DX for a while, and found this article on connecting it via Wi-Fi.
I used that idea, but with a Hame-A1 clone, that has a built in battery.
It works OK. But, if your signal drops out, the camera doesn't retry the send operation, so it only works while you have an uninterrupted Wi-Fi signal.
I also experimented with the 1DX connected to the Hame-A1, with that Wi-Fi tethered an iPhone. With this I could connect, over the internet, to a remote FTP server. That was even more intermittent, and ment leaving an FTP server on the internet somewhere. I really hate FTP because the passwords are sent in plaintext over the internet, which isn't great in this day and age.

What I wanted

I figured that if I replaced the Wi-Fi adapter with something more intelligent, then I could run an FTP server on the device. With the FTP server local to the camera, then it should always FTP OK (it's over the ethernet cable). The device could store the data locally, and then forward it to where I wanted, when it had a connection. It could use a secure protocol to do that, and it could retry if there were communication problems.

OpenWRT

I've known about the OpenWRT project for a while. I used their replacement software on my home router to do a few advanced things (and also to get rid of the local language the routers came with).
I looked at re-flashing the Hame clone with it. To do this you need to open it up, find the serial port connections, and attach a special serial cable. While I was checking where the connections were on my board, I took a close look at the RAM chip. My model only had 16M of RAM. At this point I decided not to upgrade it to OpenWRT, as with only 16M of RAM things are quite limited.

Alternatives to the Hame-A1

So I looked on the OpenWRT supported devices list and found the TP-Link MR3040. It is similar in concept to the Hame-A1, but it has more RAM, a slightly bigger battery, but most importantly, you can reflash it with OpenWRT from the web GUI, so you don't need any special leads or to take it apart.
I got mine from pchome in Taiwan for the delivered price of NT$1299, which was about US$43, £26.

Basic Steps

This is free advice, that's the limit of my liability.
These are my notes, they worked for me, and they should work for you. If you mess up the config, then follow the OpenWRT recovery procedures, hopefully you will only need the Failsafe mode that on the MR3040 is triggered with the reset button.
These step are based on the ones on the OpenWRT MR3040 page.

Download the software. Read the serial number label to check the version you need, mine was a V1.0 device, so I used this ROM.
Connect your PC to the LAN port of the MR3040
Boot the MR3040
From a PC browser, browse to the upgrade firmware option, and flash the OpenWRT ROM you just downloaded.
Re-boot the MR-3040
Disconnect, then reconnect the ethernet cable (to get a new DHCP address)

At this point you've got a router running OpenWRT! You can use it in the same was as the Hame-A1, but with an easier to use interface.

Advanced Steps

For this you need a Linux PC, preferably connected to the Internet, with ssh on a public internet port. For the local FTP server, you need a USB memory stick.

Basic setup

With your PC connected to the lan port of the MR3040.

Browse to 192.168.1.1 (the OpenWRT default address)
Login to LuCi (the web interface).
Initially the root password is blank, so just press return
Then use "Go to password configuration" to set a password (remember to save & apply)
Go into the Network/wifi/scan, select your Wi-Fi network
Set the Wi-Fi security details. (remember to save & apply)

At this point your PC should be able to access the internet via the MR3040, using it as a "Ethernet-to-Wi-Fi"/"Game adapter".

Installing a few packages

At this point, the in the System/Software menu, my device was reporting "876.00KB free".

Update the list of available packages
Search for, select available packages, then install the following:
1. kmod-usb-storage
2. kmod-fs-ext4
3. vsftp
4. rsync
#1 and #2, allow us to mount a USB ext4 formatted USB memory device.
#3 is the "Very Small FTP" server
#4 is the rsync package, that will allow us to sync files over a secure connection

At this point I had 356K of ROM remaining.

Preparing the USB key

I used a Sandisk Cruzer Fit CZ33 USB device, it's only a little bigger than a USB connector, and with 32G it has plenty of space.
On the Linux PC I used fdisk to partition it, then formatted it as ext4.
Make sure your Linux PC has the following installed:

rsync
sshd

Configuring the MR-3040

From your PC, log in via SSH (to 192.168.1.1 as root, with the password you set earlier). I use Putty on a Windows PC.
Plug in the USB memory stick to the USB port on the MR3040, then from the root account on the mr3040, type in the following commands.
In vi, use "i;" for insert, ESC to change mode, ":q!" to exit without saving, "ZZ" to save/exit.

mkdir -p /home/ftp/usb
touch /home/ftp/usb/NOT_MOUNTED
vi /etc/config/fstab
# Device Mountpoint FStype Options Dump Pass# /dev/sda1 /home/ftp/usb ext4 rw 1 1
/etc/init.d/fstab enable
vi /etc/vsftpd.conf
background=YES listen=YES anonymous_enable=YES local_enable=YES write_enable=YES check_shell=NO session_support=NO anon_upload_enable=YES anon_other_write_enable=YES anon_mkdir_write_enable=YES
reboot

At this point your MR3040 has a 32G filesystem, that is accessable from the FTP server!

Connecting the MR3040 to the 1DX over FTP

For this I find it easier to disable DHCP, and use static IP addresses. My home network uses 192.168.1.X, and the DHCP server only assigns 1..200, so I use 201+ for static assignments. Assigning the router a static address in this range means I can just plug it onto my network, and log into it, to clean up the file system, back it up, or just edit the scripts.
Using a static IP address, for the server and camera, means that there are no DHCP timeout/retry issues. So you can turn the camera and mr3040 on and off without having to worry about waiting for the DHCP to have happened.
Now is a good time to switch the Wi-Fi to use your mobile phone tethering function (you do have unlimited data plan don't you?).

With the mr-3040 connected to your PC browse to http://192.168.1.1
Turn on Wi-Fi tethering on your mobile phone
Go into the Network/wifi/scan, select your phone
Type in the security details, and connect to it (remember to save & apply)
In the Network/Settings
- disable the DHCP server
- Set a static address of 192.168.1.222
- Save & apply

In the 1DX communications menu, set the following options:

Communication mode: FTP trans
In the setup menu:
- Automatic transfer: enable
- Picture format: small-JPG
In the LAN settings
- LAN type: Wired
- IP address: 192.168.1.221
- Subnet mask: 255.255.255.0
In the target server:
- Address: 192.168.1.222
- Port No.: 00021
- Login name: anonymous
- Password: {blank}
- Target folder: /usb
- Passive mode: Enable

At this point, after you take a photograph with the 1DX, it should upload it to the USB stick on the MR3040.
To test this, take a test shot. You should see the LAN light flash (it's quick, normally less than a second). Check that the communication error menu is greyed out. If it's not then check the error message and try to work out what's wrong!

With the static addresses, I can just plug the mr3040 and the camera onto my home wired network for testing/config, my PC will use the main network DHCP server, and can access 192.168.1.222 without having to assign an address. If you directly connect your PC to the mr3040, then it's not running a DHCP server, so you will need to manually assign an address to you PC.
Having the camera and mr3040 connected to your home network, is good way to monitor what is happening.
When the 1DX uses FTP to upload the photos it will create a folder tree in the FTP area on the mr3040. It will start with an "A" or "B" directory (depending on if your read default is card1 or card2), then a DCIM folder, then a camera specific folder. That folder should contain the uploaded photos.
If you ssh into the mr3040 (as root on 192.168.1.222), you can look in the /home/ftp/usb folder and check what is created.

In normal use you can directly connect your 1DX and the mr3040 with an ethernet cable.

Copying files to your remote server

Assuming your Linux machine is on the internet on a public port, you can use rsync to upload the photos to it.
You will need the following information:

{your server port} - Default is 22, but probably best to use something else
{your server username} - Something other that root, as you'll probably want to block root logins
{your server name} - Either a public IP address, or maybe {something}.duckdns.org
{your server targetdir} - The folder on your target server that you want the files to be copied to

Here are the next steps:

ssh into the mr3040 as root (remember it's on 192.168.1.222 now)
Create public/private keys for the root account on the mr3040.
- dropbearkey -t rsa -f ~/.ssh/id_rsa
- dropbearkey -y -f ~/.ssh/id_rsa | grep "^ssh-rsa " > /tmp/export_keys
Copy the mr3040's public key to the Linux machine
- scp -P {your server port} /tmp/export_keys {your server username}@{your server name}:/tmp/
- # Login using the password for {your server username}@{your server name}
Add the mr3040's public key to the Linux machine's "authorized_keys" file. This allows the mr3040 to log into the Linux machine with using a password all the time.
- ssh -p {your server port} {your server username}@{your server name}
- # Login using the password for {your server username}@{your server name}
  - cat /tmp/export_keys >> .ssh/authorized_keys
  - mkdir -p {your server targetdir}
  - exit
Check the public key works, and tidy up
- ssh -p {your server port} {your server username}@{your server name} -i ~/.ssh/id_rsa
- # This shouldn't prompt for a password, if it does, check the previous few lines.
  - rm /tmp/export_keys
  - exit
- rm /tmp/export_keys
Create a backup script on the mr3040
- vi ~/backup.sh
  #!/bin/sh # Setup variables, based on your server info DEST_PORT={your server port} DEST_SERVER={your server username}@{your server name}:/{your server targetdir} # Setup the target dir on the mr3040 SRC_DIR=/home/ftp/usb # Make sure the file at least exists touch /tmp/xfer.old # In the unmounted directory, we created a file called NOT_MOUNTED if [ -e $SRC_DIR/NOT_MOUNTED ]; then # If it exists then we need to mount the usb key echo Warning: backup.sh did not find mounted filesystem mount $SRC_DIR elif [ -e /tmp/xfer.now ];then # If the xfer.now exists, then we're probably still running from a minute ago echo Warning: backup.sh found /tmp/xfer.now, assuming previous call still running else # SRC_DIR mounted, and we're not already running, dump the source file sizes du -a $SRC_DIR/* | egrep -i "JPG|CR2|MOV" > /tmp/xfer.now # Has anything changed since last tim we did the transfer? cmp -s /tmp/xfer.old /tmp/xfer.now if [ $? -ne 0 ];then # Yes, something changed. For debug date stamp when we started. date > /tmp/xfer.done # They do the transfer, use --size-only to speed up the checking /usr/bin/rsync --size-only -e '/usr/bin/ssh -i /root/.ssh/dropbear -p $DEST_PORT' -av $SRC_DIR/ $DEST_SERVER/ >> /tmp/xfer.done if [ $? -eq 0 ];then # If the rsync didn't fail, then move the current file sizes to the old list mv /tmp/xfer.now /tmp/xfer.old fi fi # Whatever we did, remove the xfer.now file, so that we can re-run next time rm -f /tmp/xfer.now fi
Add this script to the cron table, to be run every minute.
- chmod +x ~/backup.sh
- crontab -e
  * * * * * /root/backup.sh

At this point, if everything is working, you can take a picture on the 1DX and on the next minute cron will run the backup.sh script. That will detected the new picture(s), and send it/them to your Linux server!

I've found that the battery life for the mr3040 is pretty good, I tend to leave it switched on, and after 4-5 hours it's still fine. You can tell from the Wi-Fi, and Ethernet lights if it's busy. So you can work out when it's safe to switch off.
It's the battery life of my phone that is the problem! 4 hours of Wi-Fi + tethering will use most of the battery. So for that I use a USB battery pack.

To do

Currently on my internet connected Linux PC, I generate some thumbnails, and do other processing to create weblinks from the thumbnails to the full-res versions. This means I can preview them in near real time.
But when the mobile connection is slow, uploading the full size files can take a while. The backlog can take some time (I've seen over an hour). So I really want to create the thumbnails and reasonable-res versions on the mr3040, then upload those first. I will then quickly get get thumbnails, and reasonable-res versions, with the full-res versions available as bandwidth allows.