Broodminder - Remote Data / Android

I'll be explaining the mobile sim card data setup we use on our hives. As a hobyist we want to keep ongoing costs as low as possible and this took many days of research around sim card data providers. With cost being the main driver the amount of data sent and recieved was the challenge to lower. This took quite a bit of engineering, involving rooting phones, setups of firewalls and very specific android applications.
If you wish to see real-time (hourly) data with the Broodminder ecosystem without being an engineer you will find the options quickly get expensive.
Broodminder CELL (with Solar) + 4G upgrade costing AU $868 (as of Aug 22). This wasn't in my plans with a low number of hobby hives, especially with ongoing costs of subscriptions and SIM card.
Broodminder CELL (without Solar) is a Thingy-91 and costs around AU $350 (as of Aug 22) which is actually not too bad and comparable to buying a cheap Android phone, with with a lot less setup hassle. Apparently it comes with a SIM card and your yearly subscription covers the cost of data but I'm yet to know if this is true in Australia. Yearly costs begin to climb with this device, around AU $80 (as of Aug 22) you'll pay for the privilege. That is in addition to your premium membership of AU $80 (as of Aug 22) per year.
Phone Hub Mode
Broodminder Apiary App has an option to run in a Hub Mode. You install the App on an old Mobile phone and the Bluetooth connects to the SubHub and pulls the data every 10 minutes. Hourly updates per sensor are then uploaded.
Where to get data
Here in Australia we have a few options:
- Mobile Network Providers - Telstra, Vodafone/TPG & Optus.
- MVNO's based on the above carriers.
- Data only IOT SIM providers.
Mobile Network Providers
Here in Australia the big guys are a total rip off and I dont see why anyone uses them for their own private or work phones. No thanks.
- Telstra ($58 / month for 40GB)
- Vodafone/TPG ($58 / month for 40GB)
- Optus ($58 / month for 40GB)
MVNO's
These companies (Mobile virtual network operators) piggyback off services from Telstra/Vodafone/Optus and they provide much better value for money with full phone call/sms and data services. These plans for forever changing so you will need to do your own research. Check out GadgetGuy's cheap sim plans.
- Kogan Mobile ($120 / year for 120GB - Vodafone/TPG)
- Coles Mobile ($99 (usually $120) / year for 60GB - Optus)
- Amaysim ($99 (usually $120) / year for 60GB - Optus)
- Aldi Mobile ($95 / year for 30 GB - Telstra)
What I don't understand is the data only plans for more expensive than the fill SIM plans and your receive less data and no SMS or calls. Strange world.
IOT's
These companies also piggyback off services from Telstra, Vodafone/TPG & Optus and provide data and SMS services. Usually they charge a small fee per month and change per MB of data.
- Hologram.io ($15 sim card, ~$1 per month + ~$0.28 / MB)
- M2M One
- Things Mobile
The Choice
Before I had all the devices installed and running I estimated the data usage and I was was really low balling it. I hoped it was sending only the bare minimum, compressed and not requesting data back. Having worked with API's and done research into MQTT, I decided I would be able to get by on a very small amount of data. I was wrong.
Firstly I chose hologram.io. These guys have a great setup and they run on all the major networks in Australia and international roaming too if you need that. Depending where you are the network connects to the best available.
I would have loved to stay with these guys if I only needed a few MB of data but I needed a lot more than I expected. I topped up with $30 and I chewed through that in no time. This was not the option.
I moved on to an MVNO. Kogan to be exact. I knew from my visits down south that Telstra and Vodafone had good coverage in the area so those were my options.
Data Usage
I have 4 devices transmitting data.
- SubHub (Temperature, Battery)
- TH2SM x 2 (Temperature, Humidity, Battery)
- W3 (Temperature, Weight, Battery)
I've pulled the data from my provider and my average data usage is around 6 MB/day. Extrapolated to roughly 180 MB / month, or 2100 MB / year.
This type of data is nowhere to be found on the Broodminder documentation and the support team also had no idea how much data was to be expected. This made it quite difficult to make a decision which provider to use.
Note that my phone device is completely firewalled to prevent any other apps using data. Gapps (Google Applications) are removed. See below for specifics.
These are higher data usage IOT devices.
The software hub
When you pay for premium with Broodminder a new virtual device appears in MyBroodminder. This, I assume, is usually where the data is shown for the Broodminder CELL. When you use it in hub mode you see an empty chart with a grid below showing last time data was received from each device.
Running a mobile phone (Android) with the Apiary app does have some catches.
- The app does not run as a background service and therefore the screen needs to stay on the whole time with the app in the foreground.
- Due to the fact above the phone needs to be plugged into power.
- Things can go wrong and the app can crash and close.
- The data network can drop out or Bluetooth lose signal and data points will go missing from the live dataset.
The Apiary app has also been obsoleted. The Bees app replaces it but does not have a Hub Mode. You must use the old app to keep using the Hub Mode. Mike tells me that the Hub Mode may be spun off into another application. This would be fantastic if so.
Android Phone Setup
The biggest problem with the Apiary app its run in the foreground and the device needs to stay awake for it to be responsive. I have implemented a lot of different applications in order to try and keep the device running and stable.
- Rooting the Android Phone
Rooting a device allows you to modify data inside the system partion such as firewall rules. The firewall app below can use these extra permissions to perform far better lockdowns of the data traffic.
As I have a rooted phone with a bare basic Android version. I did not install any Google Applications (gmail etc) as there is no need on this device. The device isn't logged into a google account either. They just use network traffic and waste resources on the device.
To install android apps I download the apk directly through the browser or install it directly using ADB. ADB is Android Debug Bridge which allows you to connect a PC to the USB of the phone and issue commands from the Computer.
By rooting the phone you can also edit files in the system partion such as the loading files for when a phone is plugged into power. Then you can call other commands such as reboot. So by power being applied to the phone from the charging cable Android will boot. This is very handy if the phone dies somehow as it does not auto start again. I have this problem as I write this and I cannot just pop down 3 hours to press the power button on the phone. I will implement this when I am down next.
Depending on the device you can use:
New Phones (fastboot)
fastboot oem unlock (allow to unlock, as promoted)
fastboot oem off-mode-charge 0
Older Phones
You may need to dig around on the internet to find a solution specific to your device. I have an old HTC and I need to implement something like this https://android.stackexchange.com/questions/39899/auto-boot-when-charger-connected-for-htc-magic
This is handy for power outages when the phone battery has died while the power was out. I have yet to implement this but I'm guessing the script will wait for the phone to be charged a little before booting to stop it dying on boot and get stuck in a loop.
Firewall / Network lockdown AfWall+
I only allow the Keep alive app and the Apiary app to send and receive data.
Autostart and StaY! AutoStart
When the phone is rebooted the Apiary app does not open by default. You will need to use a 3rd party app to do this. You can also use apps to monitor if an app is open and restart it.
This app was in the play store but is not longer available there.
Keep Alive App Network KeepAlive
The mobile network is not strong where the hives are so i use an app to keep the mobile network alive.
Automate App Automate
Sometimes errors occur with all software or the network so rebooting the device automatically will help fix any issues that may happen outside of an app and keep the device fresh.
I wrote a basic workflow which checks if the time is between 10 and 11:30 pm and it will reboot. Then it waits 1 hour before checking the time again. I originally has this rebooting twice daily at 10 am as also but this is probably overkill.
I have also been thinking about a way to sync all the data one a day from the Sub-hub as sometimes data points are missing due to connectivity issues between the sensors. Usually when I or a family member visits the hives they manually sync from Sub-hub and the points are added. I want to automate this to a weekly task from the Phone hub. One thing to note about this though is it throws out all the data in Mellisphera and that sometimes take 2 days to get back into sync.
ZeroTier ZeroTier
I run VPN software so I can remotely connect when required. I plan on adding additional apps to remotely control if required.
- Setting the data network type.
As of now (2022) the 3G networks are still available in Australia and won't be shutdown until 2024. I have set my data network to specifically use 3G as it's more stable for my location. 4G is faster but has more dropouts and we rather stability of data over speed for what we're doing. I will need to change this later...