Warning: Attempt to read property "ID" on null in /var/www/blog/wp-content/themes/c2012/functions.php on line 39

Warning: Cannot modify header information - headers already sent by (output started at /var/www/blog/wp-content/themes/c2012/functions.php:39) in /var/www/blog/wp-includes/feed-rss2.php on line 8
Sergey's blog https://blog.sergem.net/ have to publish, sometimes Fri, 16 Feb 2024 01:03:59 +0000 en-US hourly 1 https://wordpress.org/?v=6.1.6 Moving from I3 to macOS with Minimum Pain https://blog.sergem.net/moving-from-i3-to-macos-with-minimum-pain/ https://blog.sergem.net/moving-from-i3-to-macos-with-minimum-pain/#respond Fri, 16 Feb 2024 00:59:22 +0000 http://blog.sergem.net/?p=362 I was a long time Linux user, most recently using I3 (Ubuntu/Regolith). The moment Apple released M1-based laptop, though, I wanted one. Because it’s fast and deservedly so. The article linked above talks about the M1 architecture and it’s hardware accelerators, which explains why any Linux kernel that would use that hardware platform to full […]

The post Moving from I3 to macOS with Minimum Pain appeared first on Sergey's blog.

]]>
I was a long time Linux user, most recently using I3 (Ubuntu/Regolith). The moment Apple released M1-based laptop, though, I wanted one. Because it’s fast and deservedly so.

The article linked above talks about the M1 architecture and it’s hardware accelerators, which explains why any Linux kernel that would use that hardware platform to full potential is very far away, so it looks like the only way to enjoy it today is to live with macOS.

Moving to macOS from I3? Dock? Ugh. Poking with a mouse moving and resizing windows? Thanks, no thanks.

Can macOS be made to work more like I3? Yabai, Amethyst – not slick. Rectangle? Way too much bother. Could a third solution be found, satisfying the requirements, simple and easy to use, setup and maintain?

The Requirements

I want maximized windows, easy keyboard-based app switch, keyboard shortcut to send to another screen, split screen occasionally.

The solution

  • Hide Dock
  • Launch apps via Command + Space
  • Maximize all windows
  • Send to another screen button
  • Split screen vertically

Hide dock / Launch apps via Command + Space

Hiding the dock is very easy – just hit Option-Command-D.

Switch between apps using Alt-Tab (and Shift-Alt-Tab).

Command+space for launching apps? It usually opens spotlight search. This could be changed in Settings: Keyboard -> Shortcuts -> Show Launch Pad: Command+Space. You might have to remove spotlight search shortcut by unchecking “Show Spotlight search” first. Note that the “3-rectangle/F3” keyboard button opens Launch Pad too, but we will use this button for more important things in the next section.

Maximize all Windows / Send to another screen button

Use Hammerspoon script to make the “3-rectangle/F3” button move the selected app to another screen (if such screen is present) and maximize it (always). At some point all of the apps are maximized and hitting Alt-Tab simply replaces one with another.

Why not use macOS-native maximized “full screen” workspaces, similar to I3? There are some issues with that and no benefits. First, the number and order of these workspaces change when the app windows open and close, so they can’t be precisely addressed with I3-like workspace shortcuts. Second, switching these workspaces triggers heavy animation. And it does not give you much extra real estate either: the “full screen” maximization does not include the top line with the camera notch on newer MacBooks – the area just becomes black and is not being used by the app window.

Split screen vertically

A small addition to the above Hammerspoon script lets us park a selected window to the left or right side of the screen using Control-Option-Command+Left or Right arrow button.

The Hammerspoon Script

function moveToNextScreen()
	hs.window.animationDuration = 0
    local win = hs.window.focusedWindow()
	screens = hs.screen.allScreens()
	local max_screen_index = 1
	local this_screen_index = 1
	for i, screen in pairs(screens) do
		max_screen_index = i
		if screen:name() == win:screen():name() then
			this_screen_index = i
		end
	end
	local next_screen_index = this_screen_index+1
	if next_screen_index == max_screen_index+1 then next_screen_index = 1 end
	local next_screen = screens[next_screen_index]
	win:moveToScreen(next_screen)
	win:maximize()
end
hs.hotkey.bind({"cmd","alt","shift"}, "S", moveToNextScreen)

hs.hotkey.bind({"cmd", "alt", "ctrl"}, "Left", function()
	local win = hs.window.focusedWindow()
	local f = win:frame()
	local screen = win:screen()
	local max = screen:frame() 
	f.x = max.x
	f.y = max.y
	f.w = max.w / 2
	f.h = max.h
	win:setFrame(f)
end)

hs.hotkey.bind({"cmd", "alt", "ctrl"}, "Right", function()
	local win = hs.window.focusedWindow()
	local f = win:frame()
	local screen = win:screen()
	local max = screen:frame() 
	f.x = max.x + (max.w / 2)
	f.y = max.y
	f.w = max.w / 2
	f.h = max.h
	win:setFrame(f)
end)

local keyboardHandler = function(event)
	local keyCode = event:getKeyCode()
	if keyCode == 160 then
		moveToNextScreen()
		return true
	end
end

keyboardEventListener = hs.eventtap.new({
	hs.eventtap.event.types.keyDown
}, keyboardHandler)
keyboardEventListener:start()

What else? Unrelated to I3: using Forklift gave me SSH mounts, something dearly missed from Linux otherwise.

The post Moving from I3 to macOS with Minimum Pain appeared first on Sergey's blog.

]]>
https://blog.sergem.net/moving-from-i3-to-macos-with-minimum-pain/feed/ 0
Fix EC2 Windows logon issue by moving the AWS image to desktop and back https://blog.sergem.net/fix-ec2-windows-logon-issue-by-moving-the-aws-image-to-desktop-and-back/ https://blog.sergem.net/fix-ec2-windows-logon-issue-by-moving-the-aws-image-to-desktop-and-back/#respond Sat, 12 Feb 2022 02:38:26 +0000 http://blog.sergem.net/?p=335 After installing a Windows Remote Desktop client access licenses, commonly known as CALs, on an AWS EC2-based Windows machine, I made a mistake of not specifying my RD license server in the terminal server settings. Once the demo licenses ran out, the CALs did not take, and I ended up with a valuable machine, used […]

The post Fix EC2 Windows logon issue by moving the AWS image to desktop and back appeared first on Sergey's blog.

]]>
After installing a Windows Remote Desktop client access licenses, commonly known as CALs, on an AWS EC2-based Windows machine, I made a mistake of not specifying my RD license server in the terminal server settings. Once the demo licenses ran out, the CALs did not take, and I ended up with a valuable machine, used by 4 people daily, locked out and impossible to login into.

The machine was running Windows 2008R2, and the online recipes of “just do mstsc /v:server /admin” did not pan out, the logins were locked for good. Now AWS does not give you any access to the console, which would have allowed me all of 2 minutes to resolve the issue.

What I did was moving the instance to my desktop VirtualBox, making the change and moving the instance back. Here is how to do it:

  1. in EC2 console, stop the Windows machine and detach the volume
  2. create a small Linux instance, my preference is Ubuntu, as things are just easier there. I used a large drive here, to keep the image of the Windows partition, it’s a throwaway disk and instance anyway. (And there also seem to be and untested way to do it without temp storage altogether); then start it.
  3. install ntfsclone (sudo apt update; sudo apt install ntfsclone)
  4. create a small local VirtualBox Linux instance on your desktop, with one small (like, 5GB) drive for Linux, and another of exactly the same size as the Windows machine; start it. You will need a Linux iso, e.g. Ubuntu Server
  5. install ntfsclone (sudo apt update; sudo apt install ntfs-3g)
  6. Attach the windows volume to the EC2 Linux machine; on EC2 Linux machine:
  7. Find the drive by typing lsblk
  8. save the first 5MB to have an exact copy of all boot sectors and partitions:
    1. sudo dd if=/dev/xvdf of=head.bin bs=1M count=5
  9. give ubuntu user a password (run “passwd”) and enable password logins (“sudo vi /etc/ssh/sshd_config”, -> PasswordAuthentication yes – restart ssh: “sudo service ssh restart” )
  10. save the Windows partition to file:
    1. sudo ntfsclone –save-image -o – /dev/xvdf1 | gzip -c > part.img.gz
  11. to skip the need for password, copy AWS .pem file to VirtualBox instance to ~/.ssh, associate it in ~/.ssh/config file:
    Host echostname
    Hostname 13.56.12.156
    User ubuntu
    IdentityFile ~/keys/q.pem
  12. on the VirtualBox Linux instance:
    1. scp ubuntu@<ec2hostname>:head.bin .
    2. lsblk (to see the name of the windows disk device)
    3. sudo dd if=head.bin of=/dev/sdb bs=1M count=5 (this recreates partition structure)
    4. ssh ubuntu@<ec2hostname> ‘cat part.img.gz’ | gunzip -c | ntfsclone –restore-image –overwrite /dev/sdb1 –
  13. Stop VirtualBox Linux instance
  14. Create VirtualBox Windows instance, preferably use the same number of processors and RAM, use PIIX4 as a disk controller; attach the Windows drive we created in VirtualBox Linux instance; start it
  15. It should come up without the need of any repair. The OS will update for some time, then open the login window.
  16. Do the fixes and test them; then stop the VirtualBox Windows machine
  17. Start VirtualBox Linux machine on your desktop with the new Windows disk attached and send the updated image to EC2 (you might want to delete old image there first to save space):
    1. ntfsclone –save-image –output – /dev/dsk/sdb1 | gzip -c | ssh ubuntu@<ec2hostname> ‘cat > new.img.gz’
  18. To be safe, you can create a separate EC2 volume, and <dd head.bin> on it to recreate partitions, or just overwrite the original disk, on EC2 Linux machine, run:
    1. cat new.img.gz | gunzip -c | ntfsclone –restore-image –overwrite /dev/xvdf1 –
  19. Stop the EC2 Linux instance and reattach the volume as a root device to the EC2 Windows machine (I had to manually specify “/dev/sda1” as the device name while attaching to make it take as root); start the Windows machine
  20. you should be good at this point

What can be done differently? We could run ntfsclone directly, without using intermediate files. E.g.:

ssh ubuntu@<ec2machine> ‘sudo ntfsclone –save-image -o – /dev/xvdf1 | gzip -c’ | gunzip -c | ntfsclone –restore-image –overwrite /dev/sdb1 –

and the reverse when uploading the disk.

The download/upload operations are practically at your disk/network speed, gunzip cuts it about in half. Overall, better use of time and effort than rebuilding the domain controller with Office 365 installed, printers, work software and users.

The post Fix EC2 Windows logon issue by moving the AWS image to desktop and back appeared first on Sergey's blog.

]]>
https://blog.sergem.net/fix-ec2-windows-logon-issue-by-moving-the-aws-image-to-desktop-and-back/feed/ 0
Boot Ubuntu 18.04.1 Live USB on MacPro 1,1 https://blog.sergem.net/boot-ubuntu-18-04-1-live-usb-on-macpro-11/ https://blog.sergem.net/boot-ubuntu-18-04-1-live-usb-on-macpro-11/#comments Sun, 21 Jul 2019 22:52:16 +0000 http://blog.sergem.net/?p=313 Problem – MacPro 1,1 is 32-bit EFI while having 64-bit architecture. There are other ways to boot that bypass EFI mode, but – booting in EFI mode on MacPro 1,1 allows for 2 more SATA ports (located on motherboard). To run a live Ubuntu 18.04.1 USB on MacPro 1,1, one needs to add a 32-bit EFI […]

The post Boot Ubuntu 18.04.1 Live USB on MacPro 1,1 appeared first on Sergey's blog.

]]>
Problem – MacPro 1,1 is 32-bit EFI while having 64-bit architecture. There are other ways to boot that bypass EFI mode, but – booting in EFI mode on MacPro 1,1 allows for 2 more SATA ports (located on motherboard).

To run a live Ubuntu 18.04.1 USB on MacPro 1,1, one needs to add a 32-bit EFI boot loader, here is how to do it:

Make a bootable USB with Startup Disk Creator on Ubuntu

Download Ubuntu 18.04.1 iso and use Ubuntu “Startup Disk Creator” or see  “Making a bootable USB manually” below. You can try booting from this drive, but it won’t work.

Use fdisk to delete EFI partition and create a new, larger EFI partition

Kernel and initrd would have to be present on EFI partition, so we need to delete existing EFI partition that is too small, and make larger one:

$ sudo fdisk /dev/sdb
d (delete)
2 (partition 2)
n (new)
2
primary
t (change type)
2 (partition 2)
ef (efi)
p (print the resuts)
w (write and exit) or q to cancel
$ sudo mkfs -t vfat /dev/sdb2
(eject and reinsert)

Copy grub, kernel+initrz and create a config file

Use “Disks” app in Ubuntu to mount EFI partition (it won’t mount automatically upon USB drive insertion, just select the partition and click little triangle “play” icon).

Create folders /efi/boot and /boot/grub on EFI partition

Put a Grub2 bootia32.efi file, taken for example from github to /efi/boot.

Copy vmlinuz (kernel) and initrd.lz (initial ramdisk) from Ubuntu iso /casper folder to /boot/grub

run blkid and note PARTUUID for sdb1 (your first USB partition with Ubuntu Live USB)

Create a text file “grub.cfg” with the following contents, replacing string after PARTUUID= with your PARTUUID above:

menuentry "Linux" {
 fakebios
 linux /boot/grub/vmlinuz file=/cdrom/preseed/ubuntu.seed root=PARTUUID=D3D50768-F8DC-4FCA-B9E8-23570817D322 boot=casper priority=low vga=normal video=efifb noefi
 initrd /boot/grub/initrd.lz
}

Previously, noefi parameter was required in linux line, but this does not seem to be the case anymore.

Boot, hold Alt key, select second EFI boot icon, the first one is trying to boot from CDROM partition. Be patient, it would take couple of minutes looking at blank screen for kernel to boot.

Finishing touches

As soon as possible, install and enable Mac Fan control daemon to avoid CPU and RAM thermal shutdown:

wget http://launchpadlibrarian.net/363285681/macfanctld_0.6+repack1-1build1_amd64.deb
sudo apt install ./macfanctld_0.6+repack1-1build1_amd64.deb

install SSH:

sudo apt update
sudo apt install ssh
passwd ubuntu

Making a bootable USB manually

  1. Create a GPT parition table, using gparted, for example
  2. Add an ext-4 volume, e.g. 3GB, copy all files from Ubuntu LiveCD ISO
  3. Add a 500M vfat volume, change its type to EFI, follow the instructions above

 

The post Boot Ubuntu 18.04.1 Live USB on MacPro 1,1 appeared first on Sergey's blog.

]]>
https://blog.sergem.net/boot-ubuntu-18-04-1-live-usb-on-macpro-11/feed/ 28
Monitoring Backyard Soil with Xiaomi Mi Flora Plant Monitor, InfluxDB and Grafana https://blog.sergem.net/monitoring-backyard-soil-with-xiaomi-mi-flora-plant-monitor-influxdb-and-grafana/ https://blog.sergem.net/monitoring-backyard-soil-with-xiaomi-mi-flora-plant-monitor-influxdb-and-grafana/#comments Sun, 20 Jan 2019 22:11:46 +0000 http://blog.sergem.net/?p=304 Xiaomi Mi Flora plant monitor is a cool high tech device for measuring plant’s soil moisture, and fertility, as well as light and temperature. The data is accessible via Bluetooth using the mobile app. The device is waterproof, the battery is said to last at least a year, and it costs about $17 on Amazon […]

The post Monitoring Backyard Soil with Xiaomi Mi Flora Plant Monitor, InfluxDB and Grafana appeared first on Sergey's blog.

]]>
Xiaomi Mi Flora plant monitor is a cool high tech device for measuring plant’s soil moisture, and fertility, as well as light and temperature. The data is accessible via Bluetooth using the mobile app. The device is waterproof, the battery is said to last at least a year, and it costs about $17 on Amazon or Ebay.

Why the project? There are issues with the mobile app – no way to share the readings with your significant other, no comparison charts between multiple sensors, and the app has to be manually synced up from time to time.

Reading sensors automatically and displaying the data through the super-flexible Grafana dashboarding tool makes the investment in plant monitors to go so much farther.

charts

Setting up your own backyard soil monitoring dashboard

You will need a Raspberry Pi Zero W, or two, if you want to monitor both backyard and frontyard, as Bluetooth Low Energy signal does not reach very far. A Raspberry Pi can pull backyard device data from a yard-facing windowsill, but not from across the house.

Amazon has them with a power supply, case and an SD card for about $33 shipped. Make sure you get the “W” version, as the other one has no Bluetooth or Wi-Fi.

You will need to install:

  1. Raspbian OS
  2. InfluxDB measurement database engine
  3. Grafana dashboarding tool
  4. Polling code from https://github.com/sergem155/miflora-influxdb

Installing Raspbian OS

Refer to official guide here and, to enable SSH and Wi-Fi, Stackexchange recommendation here

Once complete and booted, log in using SSH, username pi and default password raspberry.

Installing InfluxDB

Great reference article is here. One thing to note it that Pi Zero W shares CPU architecture with Raspberry Pi1, not more recent Pi2 or Pi3.

On Raspberry Pi:

$ sudo apt-get install apt-transport-https curl
$ curl -sL https://repos.influxdata.com/influxdb.key | sudo apt-key add -
$ echo "deb https://repos.influxdata.com/debian stretch stable" | sudo tee /etc/apt/sources.list.d/influxdb.list
$ sudo apt-get update
$ sudo apt install influxdb

edit /etc/influxdb/influxdb.conf

$ sudo service influxdb start
$ influx
> CREATE DATABASE plant_monitors

Installing Grafana

On Raspberry Pi:

$ sudo vi /etc/apt/sources.list.d/grafana.list
$ echo "deb https://dl.bintray.com/fg2it/deb-rpi-1b stretch main" | sudo tee /etc/apt/sources.list.d/grafana.list
$ sudo apt update
$ sudo apt install grafana

edit /etc/grafana/grafana.ini

$ sudo service grafana-server restart

Installing polling code

On Raspberry Pi:

$ sudo apt update
$ sudo apt install git
$ sudo apt install python3-pip
$ sudo pip3 install bluepy
$ git clone https://github.com/sergem155/miflora-influxdb.git
$ cp config-example.py config.py

Edit config.py to reflect the addresses of your plant monitors and the location of your influxdb.

Use sudo scan.py to detect new plant monitors. Hint – to avoid confusion, turn them on one by one between scans and mark them with a sharpie 🙂

Set up to run poll-insert.py from cron one a day

You will also need to make sure your Raspberry Pi’s Bluetooth supervision timeout is set to  at least 1sec (1000ms), otherwise the connection might get interrupted by Pi’s controller while the plant monitor is in deep thought. You will want to update your Bluetooth supervision timeout periodically (it tends to reset itself)

$ sudo crontab -e

50 * * * * /home/pi/miflora-influxdb/set-supervision-timeout

Making dashboards

Go to grafana, e.g. http://flowers:3000/ in my case.

Create a new dashboard.

Edit query. One example is below, in both GUI and text representation:

query-guiquery-text

Where the protocol comes from

The protocol between the phone app and the device is not published. Past attempts to figure it out (article) have resulted in code that still works, but only does a one-time immediate reading. It would have to be repeated periodically, arguably resulting in shorter battery life.

The app is able to pull past hourly data, so it was possible to figure the protocol by looking at Android bluetooth capture log file using Wireshark, consulting Android system log, and doing some Googling.

The code works with firmware versions 2.7.0 and 3.1.9. Warning: running your new device with a mobile phone app will most likely result in its firmware upgraded.

So how does the protocol look like?

  1. It performs a cute encrypted handshake. Without it, the device will hang up in the middle of hours retrieval.
  2. It gets device’s internal timer (seconds), as well as Pi’s timestamp as a reference
  3. It get the count of readings and then
  4. Reads hourly data, by asking to prepare the data for hour X and then reading it upon receiving a notification that it’s ready

More details are in the code

Cute handshake

The handshake, the hardest part to comprehend, consists of a fixed start command, a challenge, an encoded response and an encoded session finish command.

Upon XORing challenge and response, we can see that it the response is the challenge XORed by the same byte string that varies only from device to device. Looking at the alogcat system log from Android we can see that “generated token”, that is then sent as a challenge, is also XORed by the same byte sequence that varies only from device to device. What would I use for battery-cheap XOR-based encryption? RC4 stream cipher is the king of spreading randomness while doing very little. We just need to find the keys. They are different from device to device, so that must involve some serial number, most likely MAC.

Looking up the internets for “miio-bluetooth” from the alogcat output leads us to the couple of interesting pieces of code: one, that confirms that the key is the mix of MAC and product id, and two that shows the MAC is reversed and the key is 8 bytes long. Figuring out the order in which the bytes are mixed, having the loadable library with native functions lying nearby in the same git repository, is trivial.

The post Monitoring Backyard Soil with Xiaomi Mi Flora Plant Monitor, InfluxDB and Grafana appeared first on Sergey's blog.

]]>
https://blog.sergem.net/monitoring-backyard-soil-with-xiaomi-mi-flora-plant-monitor-influxdb-and-grafana/feed/ 2
How I Built a Thermostat for NuHeat Floor https://blog.sergem.net/how-i-built-a-thermostat-for-nuheat-floor/ https://blog.sergem.net/how-i-built-a-thermostat-for-nuheat-floor/#comments Sat, 17 Mar 2018 20:02:24 +0000 http://blog.sergem.net/?p=274 NuHeat floor thermostats are great but have a couple of drawbacks: they cost a lot and they fail from electricity spikes. After two replacements, I decided that a cost of a NuHeat thermostat is sufficient budget for making something cool while not paying NuHeat for their extortive designs. Description It’s a Raspberry PI-based GFCI-protected 16A […]

The post How I Built a Thermostat for NuHeat Floor appeared first on Sergey's blog.

]]>
NuHeat floor thermostats are great but have a couple of drawbacks: they cost a lot and they fail from electricity spikes. After two replacements, I decided that a cost of a NuHeat thermostat is sufficient budget for making something cool while not paying NuHeat for their extortive designs.

Description

It’s a Raspberry PI-based GFCI-protected 16A thermostat with 7 inch dimmable touch screen. User interface is web-based, accessible over WiFi or the touchscreen.

User Interface

Web page running on Chromium browser in kiosk mode on a 7-inch touchscreen:

bathroom_

bathroom_ 2bathroom_3

 

 

 

Mobile User Interface

Same web page, on a mobile phone:

mobile-homemobile-schedulemobile-period

Hardware

Bill of materials

Thermostat Base: (~$44.21) – I could’ve used just this, with user interface accessed using my mobile phone.

Screen: ($98) – Nice addition with touch control and weather display

Dimmer: ($6.95) – without it, the screen is bright enough to fully light the bathroom at night

Total: ~ $149.16 plus shipping. Pretty close.

Connections

I had to chisel back of the frame to widen it a bit to fit the screen. The components are attached to the back of the screen with small patches of 3M VHB double-sided sticky foam tape.

modules

All connections are from Raspberry PI GPIO pins to sensor pads named correspondingly:

Light Sensor

  • 1 – +3.3V
  • 3 – I2C SDA
  • 5 – I2C SCL
  • 6 – GND

Dimmer PWM control to HDMI Driver

  • 12 – PWM (HDMI Driver backlight control pad)
  • 6 – GND (hm, looks like 14 would have been a better choice)

Touchscreen

  • USB and HDMI connections

Note: USB cable, both ends being micro-B plugs, requires a jumper between pins 4 & 5 to indicate master device on Pi side

Also, I soldered all the brightness jumpers on HDMI driver board for maximum brightness.

Floor Sensor ADC

floor-sensor

Relay Control

relay

Power Supply and GFCI Outlet

gfci

relay-other side

Power supply is removed from its original bulky case and placed into a custom housing made by two diagonally cut pieces of a small plastic box.

Both relay and power supply are connected to the output terminal of the GFCI outlet – terminals protected by ground fault interruptor (the other pair of terminals are AC line inputs). Couple of nice benefits of connecting power supply after the interruptor: Raspberry can be reset by outlet’s “Reset” button or turned off altogether by the “Test” button.

Again, both power supply and relay are attached to the outlet with 3M VHB tape.

Software

Most appreciated feature turned out to be the current weather display and weather forecast from National Weather Service.

Base Thermostat

holdtemp.py – python script reading temperature from ADC, then turning relay on or off based on the temperature threshold in /run/lock/set-temp.txt. Writes current temp to /run/lock/current-temp.txt

restart-holdtemp.sh – shell script to kill holdtemp if it hangs (and does not update current-temp.txt for 2 minutes)

settemp.sh – shell script that sets set-temp.txt – runs from cron. Ignores the setting if the file /run/lock/schedule-status.txt is not present, this is used to disable scheduled operation while leaving schedule settings in place

Web-based Controls

/var/www/thermostat – contains php scripts for reading and writing crontable and current-temp, as well as html, css, fonts and javascript to display web ui. The UI can be used from any browser, e.g. from a phone

Screen

runkiosk.sh – script to launch X11 GUI, with Chromium browser in kiosk mode pointing to the web-based thermostat user interface URL (http://localhost)

Dimmer

dimmer.py – script that reads light sensor and sets PWM pulse width that adjusts screen backlight

Requires pigpiod to be installed and running

Setting Up

Web server config

apt install nginx php-fpm
sudo mkdir /var/www/cache; sudo chown www-data.www-data /var/www/cache
sudo mkdir /var/www/thermostat; sudo chown www-data.www-data /var/www/thermostat

unpacked website folder from github into /var/www/thermostat and updated dependencies using npm and bower (actually, just unpacked a zip file)

added nginx config file:

sudo vi /etc/nginx/sites-enabled/bathroom

with contents

proxy_cache_path /var/www/cache levels=1:2 keys_zone=my_cache:1m max_size=1g inactive=60m use_temp_path=off;

server {

    listen 80 default_server;
    listen [::]:80 default_server;

    root /var/www/thermostat/app;
    
    index index.php index.html index.htm;
    
    server_name bathroom;
    
    location / {
        try_files $uri $uri/ =404;
    }
    
    location ~ \.php$ {
        include snippets/fastcgi-php.conf;
        fastcgi_pass unix:/run/php/php7.0-fpm.sock;
    }
    
    location /weather_current {
        proxy_cache my_cache;
        proxy_pass https://api.weather.gov/stations/KSFO/observations?limit=1;
        proxy_cache_use_stale error timeout http_500 http_502 http_503 http_504;
        proxy_ignore_headers Cache-Control;
        proxy_cache_valid any 60m;
        add_header X-Cache-Status $upstream_cache_status;
    }
    
    location /weather_forecast {
        proxy_cache my_cache;
        proxy_pass https://api.weather.gov/gridpoints/MTR/91,115/forecast;
        proxy_cache_use_stale error timeout http_500 http_502 http_503 http_504;
        proxy_ignore_headers Cache-Control;
        proxy_cache_valid any 60m;
        add_header X-Cache-Status $upstream_cache_status;
    }
}

The two cached URLs above are location specific, if I lived not in SF Bay Area, I would have used URLs with other coordinates in parameters.

Set webserver to auto-start:

sudo systemctl enable nginx

It will need to be started in rc.local ac described below, though, as it fails to resolve weather service URLs above on start 🙁

Screen config

edited config.txt

sudo vi /boot/config.txt

added to force 800×480 mode and to set overscan margins where screen is hidden behind frame edges:

overscan_left=43
overscan_right=21

hdmi_group=2
hdmi_mode=1
hdmi_mode=87
hdmi_cvt 800 480 60 6 0 0 0

I installed package for running chromium in a kiosk mode:

apt install matchbox-window-manager nginx php-fpm

Enabled pigpio daemon:

sudo systemctl enable pigpio

edited crontab:

crontab -e

added temperature and relay control restart script

*/5 * * * * /var/www/thermostat/scripts/holdtemp_restart.sh

edited rc.local:

sudo vi rc.local

added (needed to make manual start of nginx because is needs resolver to be connected to the internet and working properly before it can start with proxy directives in the configuration):

sudo chmod 0660 /dev/tty*
sudo service nginx start
sudo -u pi xinit /var/www/thermostat/scripts/runkiosk.sh& 
sudo -u pi /var/www/thermostat/scripts/dimmer.py& 

Notes

Why ADC is read in differential mode with one input hanging in the air? Why ACD is read multiple times over several seconds and averaged?

It turned out that the thermistor cable picks up all sorts of noise, most of it being AC current. Having one ADC input in the air helps to alleviate this a bit, the rest is handled by averaging: when several full wavelengths of samples are captured, the phase of start and finish matters less.

Looking back any pitfalls I could have avoided?

Light sensor should have been placed on top of the frame. At the bottom it’s covered by hand when using touch screen, which then promptly dims.

The post How I Built a Thermostat for NuHeat Floor appeared first on Sergey's blog.

]]>
https://blog.sergem.net/how-i-built-a-thermostat-for-nuheat-floor/feed/ 2
How 1/9 arc sec elevations data looks in Qmapshack https://blog.sergem.net/how-19-arc-sec-elevations-data-looks-in-qmapshack/ https://blog.sergem.net/how-19-arc-sec-elevations-data-looks-in-qmapshack/#respond Fri, 16 Feb 2018 04:48:38 +0000 http://blog.sergem.net/?p=269 DEM data downloaded from here Get Qmapshack End-to-end guide

The post How 1/9 arc sec elevations data looks in Qmapshack appeared first on Sergey's blog.

]]>
Screenshot from 2018-02-15 20-37-39

DEM data downloaded from here

Get Qmapshack

End-to-end guide

The post How 1/9 arc sec elevations data looks in Qmapshack appeared first on Sergey's blog.

]]>
https://blog.sergem.net/how-19-arc-sec-elevations-data-looks-in-qmapshack/feed/ 0
Jailing IoT devices with OpenWRT https://blog.sergem.net/jailing-iot-devices-with-openwrt/ https://blog.sergem.net/jailing-iot-devices-with-openwrt/#comments Sat, 04 Feb 2017 23:39:47 +0000 http://blog.sergem.net/?p=238 Most IoT devices communicate with servers on the Internet. How to ensure they are not used for malicious purposes by the manufacturer, government, or some random hacker? My PetWant cat feeder has remote control via a mobile phone app, video and, most importantly, feed notifications reporting if the food was dispensed or got stuck as […]

The post Jailing IoT devices with OpenWRT appeared first on Sergey's blog.

]]>
Most IoT devices communicate with servers on the Internet. How to ensure they are not used for malicious purposes by the manufacturer, government, or some random hacker?

My PetWant cat feeder has remote control via a mobile phone app, video and, most importantly, feed notifications reporting if the food was dispensed or got stuck as it happens sometimes.

The device has telnet interface, contacts over 10 servers around the world, attempts to download firmware updates from a server in China. It also checks for, and, if found, executes an arbitrary script from the same server. The device is built around IP security camera Foscam, there is a good analysis on what it does.

OpenWRT is an open source firmware for many wireless routers. I run it on TP-Link AC1750.

My concerns

  1. My cat feeder has access to my home network: home server, phones, laptops, and could be used for snooping on them.
  2. It also can be used to launch a denial of service attack on somebody (check out recent Mirai botnet).

Unfortunately, I can’t disconnect it as I need feed notifications on my phone.

Jail it!

My solution is to jail it – isolate it into a separate network with Internet access and clamp its upload bandwidth to 5kbps.

Another solution сould be to restrict communication to the hosts it needs to communicate with, but the list of these hosts seems to change from time to time.

Creating a Separate Network with OpenWRT

  1. Create an additional wifi network on one of the radios
  2. Set wifi security for it
  3. Assign it an IP address
  4. Enable DHCP server in the network
  5. Allow it to communicate with WAN

All of the above is done in OpenWRT admin, with no scripts or coding.

Create an additional wifi network on one of the radios:

Login into OpenWRT, go to Network-> WiFi in main menu, click Add

wireless-overview

In Interface Configuration, General Settings, specify SSID for new wifi network, check “create” as a network connection and fill its name (I called it “lanpw”):

interface-configuration

On Wireless Security tab, specify wifi encryption and password:

interface-configuration-security

Click “Save”.

Now go to Network-> Interfaces, find the interface you’ve just created (“lanpw” in my case)

Set protocol to “Static Address”  and click “Change Protocol”

Fill in an IP address (this also defines the address of the network, so make sure to specify something different from other networks you have. I chose 192.168.5.1, with netmask 255.255.255.0 the network is 192.168.5.0) :

interfaces-lanpw

Specify the DHCP network range – click “Setup DHCP Server”, I left the defaults:

dhcp-server

on Firewall Settings tab, select “unspecified -or- create:” and fill name for new firewall zone.

firewall-settings

Click “Save”

Connect the created network to the Internet (WAN):

Select Network-> Firewall in the main menu and find your created firewall zone, “lanpw” in my case, and click Edit:

firewall-zones

Check connections to wan for source and destination forwarding. Note I left out the connection to my lan, as I don’t want my new network to communicate with it.

inter-zone-forwarding

Click “Save and Apply” and restart the router. Test the new network with a mobile phone or some other device. It will be able to talk to the Internet, but not devices in the home network.

Clamping Upload Bandwidth

  1. Create a traffic control class in QoS settings restricted to 5kbps
  2. Mark traffic from the jailed network with this class in firewall scripts

Building on the Simple QoS settings I run, I added a new class, and named it “1:40” :

$TC class add dev $IF_DSL parent 1:1 classid 1:40 hfsc ls rate 5kbit ul rate 5kbit

Ls 5kb means minimum limit when other traffic is present and ul 5kb means the limit when there is no other traffic.

In the script, add classification command – make sure to edit your interface name and IP address:

$IPTMOD -s 192.168.5.0/24 -j CLASSIFY --set-class 1:40

I also add same command in OpenWRT firewall custom rules, reloaded each time firewall config changes, this is the same command as above but all variables expanded:

/usr/sbin/iptables -t mangle -A POSTROUTING -o eth0 -s 192.168.5.0/24 -j CLASSIFY --set-class 1:40

The post Jailing IoT devices with OpenWRT appeared first on Sergey's blog.

]]>
https://blog.sergem.net/jailing-iot-devices-with-openwrt/feed/ 7
X-Fusion Hilo vs Rockshox Reverb – A Technical Analysis https://blog.sergem.net/x-fusion-hilo-vs-rockshox-reverb-a-technical-analysis/ https://blog.sergem.net/x-fusion-hilo-vs-rockshox-reverb-a-technical-analysis/#respond Tue, 29 Nov 2016 05:03:25 +0000 http://blog.sergem.net/?p=221 I just overhauled my Rockshox Reverb seatpost, and found out how it works in the process. I own an X-Fusion Hilo too, so after I finished, I got curious and watched a video of its overhaul, so I now know how both seatposts work. In addition, I have a long-term perspective, as I rode each […]

The post X-Fusion Hilo vs Rockshox Reverb – A Technical Analysis appeared first on Sergey's blog.

]]>
I just overhauled my Rockshox Reverb seatpost, and found out how it works in the process. I own an X-Fusion Hilo too, so after I finished, I got curious and watched a video of its overhaul, so I now know how both seatposts work. In addition, I have a long-term perspective, as I rode each seatpost for about a year.

Design

X-Fusion Hilo

reverb-and-hiloHilo has two chambers for oil and air with a valve that allows oil-air mix to move between them.

This explains why Hilo sometimes feels squishy – you are supported by an air cushion in the outer chamber.

This also means the top seal is under pressure and any issue there is critical to the seatpost operation.

Rockshox Reverb

In Reverb, the oil is kept separate from the air by an internal floating piston (IFP), which means the seatpost is never squishy, as at any time you are supported by a locked oil column and no air.

Main seal is never under pressure, which means any nicks on the shaft are not a big deal.

More Observations

X-Fusion Hilo

You can get most of the squishiness out of it by pumping it up and down while holding the remote open. At the end, at full extension, pump it with tiny amplitude while keeping the seatpost vertical to get the last air bubbles from the outer chamber.

Hilo is slow to compress at the end of the range – I keep wondering if I am getting the advertised 125mm.

Hilo may have very slow return. The reason is not enough air pressure – mine it was low right from the factory. When adding pressure to Hilo, it must be put upside down and pumped several times, otherwise you risk getting a stream of pressurized oil into your shock pump.

Hilo has cool joystick, which could be engaged in any direction. It uses a cable, which is easy to shorten or replace.

Hilo saddle mount creaks. This can be cured to a degree by a dry lubricant (e.g. dry moly) on the pivot joint cradle.

Rockshox Reverb

Reverb has a bit of rotational play – saddle nose could be moved side to side a bit – a small annoyance.

Reverb button seems to engage at the very last mm of its travel.

Reverb has failed on me after a year of trail riding.

Verdict

My choice of a future seatpost will be Rockshox Reverb.

It failed after a year of riding – bottom seal blew up (good description and photo of the issue) and the pressure was lost. I used blue loctite on the bottom seal’s threads after rebuild, so I have high hopes it will never happen again.

While Hilo is holding up pretty good so far, the oil-air mix design means any failure of the pressurized top seal, e.g. related to shaft abrasion, would make it to lose pressure, and, I am afraid, will not be repairable.

The post X-Fusion Hilo vs Rockshox Reverb – A Technical Analysis appeared first on Sergey's blog.

]]>
https://blog.sergem.net/x-fusion-hilo-vs-rockshox-reverb-a-technical-analysis/feed/ 0
Coffee with crema from an Aeropress – a howto https://blog.sergem.net/coffee-with-crema-using-aeropress-howto/ https://blog.sergem.net/coffee-with-crema-using-aeropress-howto/#respond Thu, 22 Sep 2016 03:55:30 +0000 http://blog.sergem.net/?p=191 What makes crema? Crema is a foam comprising coffee oils, carbon dioxide created during coffee roasting and air trapped in coffee grounds. There are several exhaustive Coffeegeek articles in the subject. (I  also think electric-pump-based coffee/espresso machines increase crema by aerating brewing water.) Why there is little crema in coffee from Aeropress? Oils and bubbles, being lighter than water, travel up when brewing. When […]

The post Coffee with crema from an Aeropress – a howto appeared first on Sergey's blog.

]]>
What makes crema?

Crema is a foam comprising coffee oils, carbon dioxide created during coffee roasting and air trapped in coffee grounds. There are several exhaustive Coffeegeek articles in the subject. (I  also think electric-pump-based coffee/espresso machines increase crema by aerating brewing water.)

Why there is little crema in coffee from Aeropress?

Oils and bubbles, being lighter than water, travel up when brewing. When Aeropress is used as described in the manual, crema is filtered last and gets absorbed in the grounds and paper filter.

crema in inverted aeropress

What’s more, absence of oils changes taste of coffee.

Inverted Aeropress Method

The solution is to brew in Aeropress inverted and extract crema first, then upend it into the cup and continue extraction in regular fashion.

this is crema in Aeropress

Using stainless steel filter is strongly recommended to minimize absorption by paper filter

I also recommend Aeropress Crema Attachment I 3d-printed on Shapeways, but you can use any suitable border you can fashion.

crema in inverted aeropress

crema in a cup

Maximizing Crema

  1. Use freshly roasted coffee beans
  2. Brew as soon as possible after grinding
  3. Do not exceed brewing water temperature (204F/95.5C)
  4. Use more coffee!

The post Coffee with crema from an Aeropress – a howto appeared first on Sergey's blog.

]]>
https://blog.sergem.net/coffee-with-crema-using-aeropress-howto/feed/ 0
Video Stabilization with Deshaker in Blender https://blog.sergem.net/video-stabilization-with-deshaker-in-blender/ https://blog.sergem.net/video-stabilization-with-deshaker-in-blender/#respond Sun, 26 Jun 2016 21:01:44 +0000 http://blog.sergem.net/?p=113 Action video is often lacking clear persistent references to stabilize video with – they either are not present in the frame for very long, or often compressed out of frame as the codec is trying to spend most of its bandwidth on moving objects in the shot. Deshaker, a plugin for Windows program VirtualDub, with its […]

The post Video Stabilization with Deshaker in Blender appeared first on Sergey's blog.

]]>
Action video is often lacking clear persistent references to stabilize video with – they either are not present in the frame for very long, or often compressed out of frame as the codec is trying to spend most of its bandwidth on moving objects in the shot.

Deshaker, a plugin for Windows program VirtualDub, with its capability to find and track multiple visual references (literally hundreds of them) automatically from frame to frame is the king of action video stabilization. (It works on Linux, too, see below).

Deshaker scanning a file

Deshaker works in 2 passes – analysis and stabilization. During pass 1, a log file is created that contains correction numbers necessary for each frame.

This file could be used in Blender as a source for transform track. Here is how:

Creating Deshaker Log

  1. Export video from Blender, audio is not necessary. It’s OK to use 50% render resolution (IMPORTANT: make sure to use the same resolution during Deshaker log import).
  2. Run VirtualDub.exe with Deshaker plugin installed in VirtualDub’s “plugins” folder
  3. Enable Deshaker with Video > Filters > Add > Deshaker
  4. Set or note location for log file, click OK – OK
  5. Open Blender video made in step 1
  6. I like to right-click and reduce input (left) video window to 25% at this point
  7. Make sure Video > Full Processing Mode is set
  8. Run analysis by invoking File > Save as AVI and setting some dummy file name. This file is discardable.
  9. Use Deshaker.log file for next step:

Using Deshaker Log in Blender

  1. Install one of 2 plugins from https://github.com/sergem155/blender-deshaker-log-import . There are 2 versions, using different stabilization algorithms (more below, but “damper” version is working well, “sinc” version is experimental).
  2. Import the plugin into Blender: File > User Preferences > Add-ons > Install from File
  3. Right-click the video (movie) track to stabilize. (It might make sense to make a metastrip out of the composition first.)
  4. IMPORTANT: make sure to use the same render resolution % used to render file for Deshaker analysis, as the log numbers are divided by that %!
  5. Import Deshaker log: File > Import > Import Deshaker log
  6. To speed things up a little, in Graph Editor, set interpolation type: Key > Interpolation Type > Linear
  7. Now you can scrub through video and see the Deshaker tracking data applied

Example output

Simple damper algorithm – the distance from center is reduced by 10% in each frame:

 

Music is Skeewiff – Blue Crystal (feat Rayna)

The black bars are kind of large, I’ve done a couple of attempts to reduce them, using a digital filter to remove low frequency motion from stabilization movement and remove “DC bias” building up. I still like simple damper better, but maybe I wasn’t playing with settings long enough, or windowed sinc’s ripples were acting up, and another algo would have been better.

Windowed sinc filter (17 points) – remove all motion < 1Hz, but with less steep roll-off:

 

Raw video without stabilization, for comparison:

Other Cool Features of Deshaker

I implemented just motion. Deshaker can also do adaptive zoom, rolling shutter rubber panning effect correction and can paint black bars with contents of previous frames, to name a few. If you’d like to code that, be my guest, the code is open.

What About Manual Correction?

It would have been awesome to put a second transform track and correct the movement a bit, but sadly, Blender’s transforms are fully applied and cropped between strips, so you get a crop upon crop:

second-transform-strip-cropping

I also tried to put the sequence as a source strip in another scene’s sequence, with a smaller viewport. While this is possible, the strip is scaled to the second composition’s size, even with image offset option turned on, so it is, unfortunately, useless too:

another-scene-scaling

Running VirtualDub on Linux

It works very nice:

  1. Install Wine Windows Emulator
  2. Install MP4 Windows codec https://sourceforge.net/projects/x264vfw/files/latest/download
  3. Enable vcrun2005 in winetricks (Wine utility)
  4. Download and unpack VirtualDub
  5. Download Deshaker and put Deshaker.vdf into virtualdub’s plugin folder
  6. Run virtualdub.exe

Update

There was a bug (sorry) in the code that was shifting all transforms 2 frames back, that is now fixed in github. (And once again, make sure to import the log with the same render resolution % you used to export it).

The post Video Stabilization with Deshaker in Blender appeared first on Sergey's blog.

]]>
https://blog.sergem.net/video-stabilization-with-deshaker-in-blender/feed/ 0