Stumbles is Ben Sturmfels

Disabling the “HP-Setup” wifi on an HP LaserJet

2025-04-07T00:00:00+10:00

Our home printer, an HP LaserJet Pro MFP M281fdw, constantly broadcasts a wifi network called “HP-Setup>40-M281”. Why does it need to do that though? We only ever plug it in by USB.

I tried every option I could find from the built-in screen on the printer to disable this wifi. I also tried a factory reset, but no luck. Turns out there's a trick.

Solution: Enable and the “Wi-Fi Direct” feature and then disable it again.

Enabling activates a new “DIRECT-40-HP M281” network and deactivates the “HP-Setup>40-M281” network. Disabling again will deactivate the DIRECT network but not re-activate the HP Setup one. Problem solved!

Talks I've enjoyed (August 2022)

2022-08-18T00:00:00+10:00

Tim Ewald - Clojure: Programming with Hand Tools (2013)

Great talk. I really connected with Tim's need for tangible, non-computing activities like woodworking — a passion I share. I'd wondered if this was all just nostalgia and romance, but Tim makes some really good points about the fundamental tradeoffs in automation.

Prefer to avoid YouTube? You can watch on Invidious.

Tim spoke extensively about his love of hand-tool woodworking and of making things (from wood or code). Known materials, an ergonomic environment and simple tools give you precision, control, flexibility and a connection to your work (and avoid dangerous dust). This takes time and effort to learn, but is worth it. You can get a lot of mileage from looking back in time and picking up tools that most people have forgotten - especially as used by people making their living prior to automation. Automation saves time, reduces costs and adds convenience, but it also trades off the beauty of proportional work for the efficiency of measurement-driven work. Automation isn't bad, but the tools significantly affect how you see the world, what is possible and the final results - there are costs to automation - eg. handmade mortise and tenon joints on doors that are strong and aesthetic compared to stub tenons that are convenient to produce with machines.

Tim made the analogy to the known material properties of Clojure data structures, the development environment (of Emacs) and the tools of Clojure functions. He highlighted that in programming too, automation is ok, but there are tradeoffs. The automation changes how you approach problems and the possible solutions you consider. When you have a killer app, everything starts to look like a problem that the killer app is the solution to. Tim worked in the Microsoft world with SOAP for around 10 years and described SOAP/WSDL as as a trainwreck driven by the fixation on automation within Visual Studio, and the assumption that it would protect them from unbounded complexity. Other tools that didn't work with with Visual Studio weren't perceived to even exist. Tim asked whether now that we use Clojure's Lein, if we know what we ship, and whether that's ok. More generally, how does automation X change how I look at the world?

Tim described programmers as most like the woodworking pattern makers — people who made unique wooden models of parts to be cast from steel. For this type of work Clojure is the perfect simple tool that gives you a broader perspective on the world.

How to fix uWSGI "OSError: write error"

2021-09-21T00:00:00+10:00

This OSError: write error issue has been frustrating me for years — ever since I began switching Python/Django web applications across from gunicorn to uWSGI Emperor. Up until recently, I didn't understand what caused these errors, whether they represented a problem worth solving, and if so how to deal with them.

Problem

I deploy Python/Django sites with an Nginx frontend server, which proxies to a Unix domain socket where uWSGI Emperor listening using the uWSGI protocol (uwsgi_pass). Something like this for Nginx:

# Nginx config (truncated for clarity)
location / {
     uwsgi_pass unix:/tmp/site_name.sock;
     include uwsgi_params;
}

And this for the uWSGI:

# uWSGI Emperor vassel config (truncated for clarity)
[uwsgi]
plugins = python3
chdir = /srv/site_name
home = /srv/venvs/site_name
module = project.wsgi
master = true
socket = /tmp/site_name.sock
processes = 3
vacuum = true
enable-threads = true  # for Sentry

A few times a week, I would see an error like below, both on my Sentry error tracking service, as well as in the uWSGI logs at /var/log/uwsgi/emperor.log:

mysite - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request / (ip 1.2.3.4) !!!
mysite - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET / (1.2.3.4)
OSError: write error

Noteably, I did not receive any Django error emails about this, only notifications from Sentry.

From my literal reading of the above, I gather that the client web browser had closed the HTTP connection with Nginx and Nginx had closed the output stream that uWSGI was trying to write the response to. Seemed reasonable enough. I hadn't received any reports from customers of issues, so assumed the issue was not causing any serious inconvenience. Still, I couldn't reproduce it, which bothered me. And while I could mark the issues in Sentry as "ignore", they still counted towards the total weekly errors report.

I tried mobile devices, command-line browsers and load testing to see if I could pin down the cause of the issue. Nothing. The errors in the logs were sporadic and unpredictable.

Over the years I made a few blind attempts to fix the issue by adjusting Nginx's uwsgi_ignore_client_abort, uwsgi_read_timeout, uwsgi_send_timeout settings and uWSGI's harakiri setting. Despite what some StackOverflow posts would suggest, none of these made any difference.

How to reproduce

I recently managed to replicate this issue. Turns out that all you need to do is open a non-trivial Django page in a regular desktop browser and hammer on the F5 (refresh) key a few times in rapid succession. Bingo!

Solution

Being able to reproduce the issue gave me confidence that the error was part of normal operation and did not represent a negative experience for the visitor. Given that the default Django mail_admins logging handler didn't report the error, it was clear that only Sentry was really a problem here.

After further testing I found that uWSGI's disable-write-exception would prevent the errors being logged by Sentry.

To prevent the (harmless) errors in the uWSGI log, you need all three of these lines in your uWSGI config:

ignore-sigpipe
ignore-write-errors
disable-write-exception

Each corresponds to one of the three error lines mentioned above.

Since applying the change, I've had no further notifications from Sentry about the issue and lovely clean weekly error reports.

An alternate fix is to just filter out these errors before they are sent to Sentry. I added the following before_send handler to the Sentry setup in the production Django settings, returning None when encountering an OSError: write error:

def before_send(event, hint):
    """Don't report "OSError: write error" to Sentry."""
    exc_type, exc_value, _ = hint.get('exc_info', [None, None, None])
    if exc_type is OSError and str(exc_value) == 'write error':
        return None
    else:
        return event

sentry_sdk.init(
    dsn='...',
    before_send=before_send,
)

Getting started with Guix deploy

2021-08-21T00:00:00+10:00

It's still early days for Guix's guix deploy, but it may well be my server deployment tool of the future. I'm quite excited!

guix deploy promises to simply and reliably drop predefined operating systems onto remote machines; including services, configuration, packages and data. In the past I've used Ansible, Salt, Puppet and Fabric for this purpose, but guix deploy has a slightly different approach. Rather than "massaging things until they look right", guix deploy uses Guix's superpowers to declare the whole operating system precisely and drop it into place. It's like functional programming for operating systems. Rather than templated YAML, guix deploy also uses a proper programming language; one of the reasons I still heavily use the Fabric deployment tool at work.

This approach could significantly reduce the maintenance burden on deploying production software. The experience should hopefully be similar to using a platform-as-a-service system but more flexible, using commodity virtual machines rather than proprietary infrastructure and without needing complicated orchestration systems such as Kubernetes.

Overall strategy

The overall strategy here is to:

Create a minimal and generic base Guix operating system image using guix system image.
Manually provision a cloud server with this base image.
Use guix deploy to replace the generic configuration with the specific configuration we want.

Note that we don't run a traditional installer at any point. In the future, guix deploy may also be able to provision the remote server for a range of cloud hosting providers. There is initial support for provisioning on Digital Ocean, but I don't use this provider.

A web server from scratch

Preparation: You'll of course need Guix installed on your local machine or a remote virtual machine. For experimentation, it's convenient to generate images and run guix deploy on a temporary remote virtual machine to save uploading lots of large files on a slow internet connection. I'd suggest provisioning a Debian VM and installing Guix. Down the track I intend to deploy directly from my laptop.

Wherever you're deploying from, create a new directory for this experiment, change into it and generate a new SSH keypair with ssh-keygen.

Create a minimal base image: Create a basic a Guix System config in vm.scm. Mine looks like this:

(use-modules (gnu))
(use-service-modules networking ssh)
(use-package-modules bootloaders ssh)

(operating-system
 (host-name "vm")
 (timezone "Etc/UTC")
 (bootloader (bootloader-configuration
              (bootloader grub-bootloader)
              (target "/dev/vda")
              (terminal-outputs '(console))))
 (file-systems (cons (file-system
                      (mount-point "/")
                      (device "/dev/vda1")
                      (type "ext4"))
                     %base-file-systems))
 (services
  (append (list (service dhcp-client-service-type)
                (service openssh-service-type
                         (openssh-configuration
                          (openssh openssh-sans-x)
                          (permit-root-login #t)
                          (authorized-keys
                           ;; Authorise our SSH key.
                           `(("root" ,(local-file "id_rsa.pub")))))))
          %base-services)))

Generate an image from this config using guix system image --save-provenance vm.scm. This will return the path of the new ~1.5GB image in the default "efi-raw" format. If you're creating the image on a remote VM you can publish it quickly by installing Nginx and building with rm -f /var/www/html/latest.raw && guix system image --save-provenance --root=/var/www/html/latest.raw vm.scm, which will then be available at http://yourvm/latest.raw.

If your hosting provider supports it, using --image-type=qcow2 is much more space efficient at ~500MB. My provider doesn't support QCOW2, but I know that Digital Ocean does.

Provision the server. After logging into my Binary Lane hosting control panel, I select "Add Cloud Server". I learned that I had to select "BYO ISO" here because it turns off their auto-magic storage resizing feature which chokes on the multi-partition Guix image; but that's probably not applicable to other providers eg. Digital Ocean. The machine will be provisioned and assigned a hostname I'm taken to the image upload screen.
Upload the minimal base image as a backup and restore it. Rather than running an installer ISO, we'll upload a full disk image. Select "Settings" (cog), "Snapshots, Backups & ISO Management". Select the "Upload" tab. Choose "Create a new temporary image", upload from "Local file" (if building locally) or "HTTP server" if building on a remote VM. Provide your image created in step 2 and select "Start Upload". After that's uploaded, select "Restore this Backup". The machine should reboot into the new image. In the recovery console, I can see the machine has successfully booted.
Test the connection to your new server. Run ssh-add id_rsa to load the key we created earlier to your keyring and then SSH to the server using the hostname or IP. In my case the generated hostname is tobacco-rebel.bnr.la, so I use ssh root@tobacco-rebel.bnr.la.
Resize the filesystem to use the full storage allocation. Our image was only 1.5G, but my provider initially allocates 20G of storage. To use this, log into the VM and run cfdisk, select the /dev/vda2 root partition, select "Resize", "Write" and type "yes". Then resize the filesystem to match by running resize2fs /dev/vda2.

Create a full guix deploy configuration in deploy.scm. My operating-system section is significantly the same as above, with additions for the Nginx web server and to authorise my locally generated packages:

(use-service-modules networking ssh web)
(use-package-modules bootloaders ssh)

(define %system
  (operating-system
   (host-name "tobacco-rebel")
   (timezone "Etc/UTC")
   (bootloader (bootloader-configuration
                (bootloader grub-bootloader)
                (target "/dev/vda")
                (terminal-outputs '(console))))
   (file-systems (cons (file-system
                        (mount-point "/")
                        ;; Must be vda2 or you won't be able to reboot after `guix deploy`.
                        ;; This is because our base image makes an EFI partition at vda1.
                        (device "/dev/vda2")
                        (type "ext4"))
                       %base-file-systems))
   (services
    (append (list (service dhcp-client-service-type)
                  (service openssh-service-type
                           (openssh-configuration
                            (openssh openssh-sans-x)
                            (password-authentication? #false)
                            (permit-root-login #t)
                            (authorized-keys
                             ;; Authorise our SSH key.
                             `(("root" ,(local-file "id_rsa.pub"))))))
                  ;; Security updates, yes please!
                  (service unattended-upgrade-service-type)
                  ;; Note that Nginx isn't automatically restarted during
                  ;; `guix deploy`, so run `herd restart nginx`.
                  (service nginx-service-type
                           (nginx-configuration
                            (server-blocks
                             (list (nginx-server-configuration
                                    (server-name '("tobacco-rebel.bnr.la"))
                                    (listen '("80"))
                                    (root "/var/www/")))))))
            (modify-services %base-services
              ;; The server must trust the Guix packages you build. If you add the signing-key
              ;; manually it will be overridden on next `guix deploy` giving
              ;; "error: unauthorized public key". This automatically adds the signing-key.
              (guix-service-type config =>
                                 (guix-configuration
                                  (inherit config)
                                  (authorized-keys
                                   (append (list (local-file "/etc/guix/signing-key.pub"))
                                           %default-authorized-guix-keys)))))))))

(list (machine
       (operating-system %system)
       (environment managed-host-environment-type)
       (configuration (machine-ssh-configuration
                       ;; Use host name or IP address here.
                       (host-name "tobacco-rebel.bnr.la")
                       (system "x86_64-linux")
                       ;; Update this!
                       (host-key "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKp5IsRNi/qU2vrWNaH9MlZnOzN4umiEXkamScuwxF4M")
                       (user "root")
                       ;; Use this key to communicate with the machine.
                       (identity "id_rsa")
                       (port 22)))))

Deploy the config with guix deploy from your local machine. Guix will initially complain about the host-key, so you'll need to update your configuration with the host key of the new machine:

$ guix deploy deploy.scm
  tobacco-rebel

guix deploy: deploying to tobacco-rebel...
guix deploy: error: failed to deploy tobacco-rebel: server at 'tobacco-rebel.bnr.la' returned host key 'AAAAC3NzaC1lZDI1NTE5AAAAIKsuev1SJ3SODqkHsRX4oWib6e6A3MiS9CArvXZhmNbq' of type 'ed25519' instead of 'AAAAC3NzaC1lZDI1NTE5AAAAIKp5IsRNi/qU2vrWNaH9MlZnOzN4umiEXkamScuwxF4M' of type 'ed25519'

After correcting the host key:

$ guix deploy deploy.scm
The following 1 machine will be deployed:
  tobacco-rebel

guix deploy: deploying to tobacco-rebel...
...
guix deploy: successfully deployed tobacco-rebel

If you run guix system list-generations on the new server, you'll see that there is now a new generation for each guix deploy, allowing you to potentially guix system switch-generations if needed.

Now create a test file on your web server:

$ ssh root@tobacco-rebel.bnr.la "echo 'Hello World!' > /var/www/index.html"

If you got this far you should be able to now access the new Nginx server, eg. For me http://tobacco-rebel.bnr.la/. Yay!

Did this work for you? Please let me know.

Further work

From here, we have a working server with Nginx, but would need HTTPS and some content to call it a website. I've added some further resources below. Adding Certbot still currently requires a dance of first deploying a config with Certbot and without HTTPS so Nginx doesn't break, obtaining the certificate, then re-deploying with the HTTPS enabled in Nginx. Certbot is run via mcron, so see herd schedule mcron for the command to initially run to obtain the certificate.

Should you upload the website content through guix deploy or separately eg. rsync? That's up to you. Many examples use rsync, but the configuration for Berlin (below) uses static-web-site-configuration which builds a static website from a git repository.

How about adding a Mumble server, or any of the other services available in Guix? They're all only a few lines of config away!

Could the base image be shrunk further? 1.5GB is rather a lot to be moving around from many internet connections. This is not so much of an issue if your provider supports compressed QCOW2 images.

My host also has a Nova API and examples of using python-novaclient and docker-machine to provision VMs. I'd like to see whether I can upload Guix images and provision new machines this way.

Wrong turns I took along the way

Here's a few issues I wasted plenty of time on:

My host doesn't support loading QCOW2 images. It accepted them, but they would always fail to boot.
Before I added the signing-key config, guix deploy would fail with error: unauthorized public key because it didn't trust the substitute packages built on my deployment machine.
I misunderstood the guix system image argument --volatile and ended up with a read-only server the regenerated it's SSH key on every reboot. Volatile in computing means non-persistent.
Resizing storage for the base image was problematic on my hosting provider due to their auto-magic resizing feature that allocates storage, resizes the partition and the filesystem. Their support advised that selecting "BYO ISO" disables the partition/filesystem resizing and you can just do it manually. Before this I did experiment with creating extra-large efi-raw images eg. 20GB, but it wasn't really feasible.
The efi-raw image creates a 40MB EFI partition at /dev/vda1 labelled "GNU-ESP", before the root partition at /dev/vda2. Deploying a config with the root partition listed as /dev/vda1 will mean the VM will no longer boot. After discussions with Mathieu (below), he is working on a single partition raw image type that will be available in the near future.

Thanks and further information

Mathieu Othacehe has a great blog post on this topic, and goes further into the web hosting configuration of the server.

Ludovic Courtès and roptat pointed out the configuration for Guix's server Berlin, which uses static-web-site-configuration.

Christine Lemmer-Webber initially encouraged me to look into guix deploy, wrote the Running Guix on a Linode Server cookbook entry and provided some examples from her own use.

Jakob L. Kreuze implemented much of guix deploy and described it in Towards Guix for DevOps and Managing Servers with GNU Guix: A Tutorial. Thanks Jakob and the other Guix contributors!

This week I learnt (Week 33, 2021)

2021-08-20T00:00:00+10:00

SVG supports SMIL animation — no JavaScript or CSS

SVG can use SMIL animation elements such as <animate> and <set> to create declarative animation/interactivity. This allows you to dynamically change attributes like position, opacity, fill and CSS class. These features seem very widely used, but would be handy for a self-contained diagram, for example a tabbed interface based on opacity.

The following demo uses the <set> element to change fill colours base on click and mouseover events, showing that events can target other elements by id.

<svg width="415" height="100" xmlns="http://www.w3.org/2000/svg"
xmlns:xlink="http://www.w3.org/1999/xlink">
<rect fill="#ff660099" x="0" y="0" width="200" height="100">
    <set attributeName="fill" to="#ff009999" begin="click" dur="1s" />
    <set attributeName="opacity" to="0.1" begin="rect2.mouseover" dur="1s" />
</rect>
<text x="20" y="55" fill="#333">Click/tap to change colour</text>
<rect id="rect2" fill="#ff660099" x="215" y="0" width="200" height="100">
</rect>
<text x="225" y="55" fill="#333">Hover targets other rectangle</text>
</svg>

Inkscape can also edit such SVGs without destroying overwriting the animation elements.

GitHub commit squashing — no need to rebase

I still occasionally get well-meaning advice from GitHub bystanders to rebase and squash my commits in pull requests. Ignore those folks — it's not necessary. Way back in 2016, GitHub introduced a pull request squash feature to give maintainers the discretion to squash a contribution into a single commit. Besides, overly-enthusiastic rebasing can cause problems for others following along with your work.

Pelican maintainer Justin Mayer wrote this in response to a piece of such advice:

(Bystander) As a general note, I would expect all the commits to be squashed into a single one, the upstream doesn't need your development history I think.

(Meyer) I'm not sure I agree. When formulated with thought and care, multiple commits can indeed be quite useful. Also, during feedback-and-change iterations, it can be useful to see what has changed in a PR over time, in which case cleaning up the commits can be done at the very end, right before merging.

(Me) I don't use GitHub a lot for collaboration myself, but I believe there's been a "merge and squash" button available to maintainers for a few years now for use at their discretion.

(Meyer) That is correct. If the commits aren't valuable on their own (e.g., if there are "fix-up" commits that add no value as separate commits), the maintainer can squash the commits if the contributor hasn't already done said clean-up.

GitLab, which I use a lot more day-to-day, has had a similar feature available for many years too. I don't use it regularly as I prefer to retain the full development history.

Reordering commits during git rebase

Reordering git commits during a git rebase --interactive is as simple as reordering the lines of text in the commit list that pops up in your editor. Although I tend not to rebase aggressively, this reordering is handy to group related changes together for easier review.

Talks I've watched (March 2021)

2021-03-15T00:00:00+11:00

Emily Ashley: 7 Falsehoods Programmers Believe about Place & Time

Emily described how the human side of collecting and visualising time-based data is more complicated than you might think, highlighting issues like varying precision where some data might have a year only, or some might be ordinal where some event happened before another but we don't know the precise time. These things don't fit well into our common date/time format. She didn't recommend any particular solutions, but mentioned that ISO 8601 does cover a range of cases.

Baishampayan Ghose: Revenge of the Pragmatists

Baishampayan discussed why adopting Clojure early in it's life worked out well for their company (100 staff). Talked about recruiting and culter and the need to grow teams rather than just hire them. Specific recommendation for all current and new Clojure programmers to read SICP. Upfront design as promoted by Clojure is different to "debugger-driven development". At scale, all problem are people problems. Small teams and FP can be very productive. FP does help build reliable software. Interestingly, he mentioned that haven't used ClojureScript because of their existing JS/TS codebase, though would like to.

Why MediaGoblin?

2021-03-11T00:00:00+11:00

We've recently released MediaGoblin 0.11.0! The release ends our support for Python 2 and significantly simplifies the maintenance of the project. It also re-introduces audio spectrograms in addition to a handful of bug fixes.

This is my second release in the maintainer seat after becoming a co-maintainer in September 2019 and sole maintainer in January 2021. Given that, I'd like to reflect on why I'm here, and some of my shorter and longer-term goals.

Personally, MediaGoblin has been a way to privately share photos of my kids with their grandparents, a place to share technical presentations and talks and somewhere to publish recordings from our free software group. But beyond this, it's also been an inspiration to me.

From the beginning, Chris and Deb have done an exceptional job of pitching a vision of a better future for artists, photographers, videographers and makers; a way for them to have independence and control over their publishing. Back in 2010, photo, audio and video publishing had recently become mainstream, but there were few good alternatives to centralised, corporately-controlled publishing platforms. This clear vision, plus the energy, deliberate community building and quirky artistic flair that Chris and Deb brought to the project, helped the concept resonate with a whole lot of people, myself included. This community of great people is a large part of why I'm here.

In many ways, MediaGoblin has already proven itself as a useful tool and a worthwhile project. Many people and organisations already use MediaGobin, so there is no doubt about the impact that we, the members of the MediaGoblin community, have made. Can we go further? Absolutely we can.

MediaGoblin had a fairly quiet period in terms of releases between version 0.9.0 in March 2016 and 0.10.0 in May 2020. Chris and Jessica diverted a lot of energy into the bigger issue of standards for federated social media through the Social Web Working Group. And even before the major interruptions of COVID-19, the Python 2-to-3 transition spread our efforts a little thin.

Despite this, we've made significant progress in the last year or so; squashing bugs, adding small features, improving documentation and testing installation on a number of operating systems. This has been partly boosted by a pilot media archiving project I've worked on with NG Media in Western Australia.

So where to from here? At this point, it would be reasonable to ask what mainstream success might look like for MediaGoblin. A lot has changed since the project was founded in 2010, but one thing that hasn't changed is the dominance of previously mentioned centralised corporate-controlled photo, audio and video publishing. There are many younger sibling projects working towards similar goals, including PeerTube, Mastodon, Pixelfed and many more. MediaGoblin does still remain unique in its focus on artwork publishing and multiple media formats.

There is still plenty of work to be done on MediaGoblin, and while there are no guarantees of imminent mainstream success, the potential up-side of and impact of our work is huge, while the potential downside is small, a concept I've recently heard referred to as "convex". This idea encourages me to keep pushing forwards, even if slowly and intermittently and not worry too much about trying to predict the future.

There are definitely easier and harder problems to solve in the world of free software. While working on programming libraries and development tools is far from simple, the scope is usually more clearly defined. End-user facing applications like MediaGoblin are challenging because they have a larger scope and are used by a wider range of people. This is part of what motivates me though — bringing free software to those beyond just the tech community.

It's been great to see the community's reaction to these recent two releases. People do still know and love MediaGoblin; both the media publishing software we have now and the ambitious federated free software platform we hope to have in the future.

MyGovID ethical, privacy and security problems for ATO Business Portal

2020-03-26T00:00:00+11:00

[Updated 16 April 2020 with response from Catherine King MP and ATO. See bottom of this post.]

I've just sent a letter to the Commissioner of Taxation about the rollout of MyGovID as the only way to log in to the ATO Business Portal. Here it is in case it helps to encourage other business owners to speak out.

Essentially the ATO is switching off the nice email/password/SMS-code MyGov login method I use to access the Business Portal to manage tax/GST/PAYG/super. The are replacing this with login via a proprietary mobile app called, confusingly, MyGovID. I'm late to the party, with the changeover due in only a few days time, but better late than not heard at all.

To: Chris Jordan AO, Commissioner of Taxation

Dear Mr Jordan

Due to the major ethical, privacy and security issues with MyGovID and its upcoming compulsory use in the ATO Business Portal, I request that the transition be deferred. This will allow for further review and revisions to MyGovID and the way it is distributed.

As a small business owner, I currently complete my Business Activity Statement, Pay As You Go tax witholding, GST and superannuation payments to employees through the Business Portal. I log in using MyGov (not MyGovID). This approach works very well for me, requiring only an email, password and SMS code.

As you know, login via MyGov is being decommissioned at the end of March 2020 to be replaced by the new MyGovID smartphone app. MyGovID has two major problems. Firstly, all business owners will require an account with either Apple or Google. Secondly MyGovID is proprietary software that business owners are asked to blindly trust and cannot audit.

To download the MyGovID app requires that the business owner register with Apple to access the Apple App Store, or with Google to access the Google Play Store. This typically requires providing full name, date of birth, phone number, address and credit card details. Apple and Google are two of the world's richest companies who's sole responsibility is to their shareholders, not to account holders. While many Australians have already given up their personal information to Apple or Google, we are really only just beginning to understand the implications of these actions. These companies have no place collecting dossiers on Australians or be in a position of trust and power between the Australian Government and its citizens.

MyGovID is proprietary software, which means that the people using it, even technology professionals like myself, have absolutely no knowledge of what it does. We can't tell what information it tracks and collects about us and whether or not it is behaving in our best interests. This is the worst kind of technology — monopoly, non-interoperable technology that we are forced to depend on and must trust on blind faith.

Personally, for ethical, privacy and security reasons I do not have or wish to have an account with either Apple or Google and choose not to use proprietary software. From April I will no longer have access to the Business Portal and will be forced manage my tax obligations by post. For my business this means not having the most up-to-date information about my tax account, spending more time on managing my tax affairs and finding an alternative method to report and pay superannuation.

As a technology professional I'm sympathetic to the challenges of designing a simple and secure online system, let alone one that is responsible for highly confidential information and is rolled out to millions of citizens. This is not easy, but it can} be done without sacrificing ourselves to Apple/Google and without putting unaccountable technology in a position of unjust power over our lives.

How could this situation be improved right now? Firstly, please defer decommissioning the existing MyGov login to allow for further public review. Secondly, please release the source code to the new MyGovID app to the public to allow it to be reviewed and verified by any Australians with the interest and technical expertise to do so. Thirdly, please ensure that the MyGovID app is available for download without requiring registration; for example in an F-Droid compatible repository.

My apologies for the lateness in raising these concern. As a busy sole-trader, it's difficult to allocate time to allocate time to these things. I would be very happy to discuss this matter with you further.

Yours sincerely,

Ben Sturmfels

CC: Catherine King MP, Federal Member for Ballarat

Update 27 March 2020: Catherine King MP responded very promptly and sent me a copy of the letter she wrote to Treasurer Josh Frydenberg about the matter on my behalf.

Update 31 March 2020: I had a lovely call from a person at ATO responding to my complaint. A couple of things they mentioned:

ATO is the first agency to use MyGovID
MyGovID has a feedback form so please use it
they have received quite a bit of feedback similar to mine
there was some form of hard deadline in place around their previous authentication set up around 10 years ago - sounded like a contract expiry but I didn't get specifics - may have been just related to AusKey
they really didn't know how the transition was going to go - now they have learned, surprise surprise, for example a bunch of tax accountants who don't have smartphones - much respect to those accountants!
currently the Digital Identity team is only speaking with people who are having technical difficulties with the app, not people who want to participate in the upstream process

All in all, they were very empathetic about the ethical issues of requiring Apple or Google accounts and trust in proprietary tech. If you can spare a few minutes, this is an important time to be heard and they are certainly listening.

Update 16 April 2020: A representative of ATO called to suggest that as a sole-trader (not a company), I can manage activity statements and superannuation through the ATO linked service on my.gov.au. I tried this and after doing the necessary linking security questions, I get essentially the exact same functionality I had via the ATO Business Portal.

This isn't an option for companies though, who are forced to use MyGovID so that multiple authorised people can access these features on the ATO Business Portal.

The representative told me that there's no plans to move my.gov.au to MyGovID login for the foreseeable future.

So that solves my issues for now, but I expect it's only a matter of time before MyGovID gets more widely rolled out.

GitLab CI/CD on OpenShift Online Kubernetes… or not

2018-06-14T00:00:00+10:00

(Spoiler) In short, it appears that you currently can't use OpenShift Online as a Kubernetes cluster for gitlab.com. I'd be happy to find out I'm wrong though.

I don't want to learn how to administer a Kubernetes cluster right now. I want less infrastracture to manage, not more. With that in mind, Openshift Online seems like the ideal Kubernetes backend to run my GitLab continuous integration/deployment services. I already pay for and use the platform, it's run by a company I trust and respect and provides a great web interface. Let's set it up!

Given an account on gitlab.com, create a new project, for example by forking minimal-ruby-app.
Given either a "Starter" or "Pro" account on OpenShift Online, create a project called gitlab-managed-apps, create a service account for GitLab to use, and copy the service token:
```
$ oc login
$ oc new-project gitlab-managed-apps
$ oc create serviceaccount gitlab-robot
$ oc policy add-role-to-user admin system:serviceaccount:gitlab-managed-apps:gitlab-robot
$ oc serviceaccounts get-token gitlab-robot
```
If you log in to the OpenShift web console and visit the gitlab-managed-apps project, you'll find our new service account under "Resources, Membership".

Alternately you can create the project and the service account via the web console, but I don't believe it's possible to view the service account token without the oc command line.
Returning to GitLab, select "Settings, CI / CD" and expand the "Runners settings". Click "Install Runner on Kubernetes". Click "Add Kubernetes cluster", then click "Add an existing Kubernetes cluster". Set the cluster name, API URL and token. For me these are something like:

Note that the "API URL" is the base URL from your OpenShift Online web console, excluding the bit after the "/" and the token is the one we exported above.
So far so good. On the following screen, under "Applications" click the "Install" button next to "Helm Tiller". If it's not clickable, try refreshing the page. If the service account is set up right, the indicator will spin for a bit, then display "Installing" for about a minute, then issue an error:
```
Something went wrong while installing Helm Tiller
/bin/sh: can't create /etc/apk/repositories: Permission denied
```
If you're fast, you can briefly see a pod being spun up and deleted on the OpenShift web console.

Unfortunately I wasn't able to overcome this issue, that's as far as I got. There's no clear indication of what actual problem is and, as far as I know, no way to dive in and find out what's going on. I suspect the issue may be related to OpenShift Online's default policy of not allowing containers to run as root, but I can't say for sure.

The disappointing thing is that back in 2016 and 2017, there did seem to be some sense of partnership between GitLab and OpenShift. There are a number of official GitLab/OpenShift videos from around this time, demonstrating the manual setup of GitLab Runner on OpenShift (pre-GitLab Kubernetes integration). Unfortunately, there don't seem to be any official and stable GitLab Runner templates for OpenShift. This official-sounding one is marked EXPERIMENTAL and this unofficial one template I came across advises that it is not compatible with OpenShift Online (only self-hosted OpenShift). The Review Apps mention down the bottom that docs for OpenShift/kubernetes are soon to be added.

The approach I have had some success with is using the .gitlab-ci.yml template that GitLab provide for OpenShift via the "Set up CI/CD" button. This uses gitlab.com's shared runners, but deploys to OpenShift. This review-apps-openshift looks promising too. But that's another blog post.

An aside: How is this meant to work?

In contrast, the GitLab CI/CD and Google Kubernetes Engine (GKE) integration appears to be getting significantly more promotion and quite deep-running integration from the GitLab team. While I don't use or recommend Google services to any of my clients due to privacy concerns and lock-in from proprietary network services, I gave this a brief trial to see how the GitLab/Kubernetes integration is meant to work.

GitLab provide a custom setup process and documentation for GKE. This mostly avoids the need to use the GKE web interface at all, aside from creating an account, creating a project and enabling API permissions. That's a good thing, since the GKE web interface seemed to be very buggy. Once Kubernetes was configured, GitLab did the rest; launching a cluster and prompting me to install Helm Tiller, GitLab Runner and create a wildcard DNS entry. Using this setup, I was able to enable the Kubernetes GitLab Runner and use Auto Deploy to successfully deploy "minimal-ruby-app" to the custom domain

The overall user experience with GKE wasn't actually significantly different to that of using the 80 line OpenShift .gitlab-ci.yml mentioned above; just that instead of using gitlab.com shared runners, the build/test/deploy work is done from on Kubernetes. The major downside of the Kubernetes approach though (either GKE or OpenShift if it worked), is that Auto Deploy magic is actually 800 lines of shell script and YAML which, sooner or later, will need troubleshooting. So while it makes for a slick webcast, I'm not sure this is a path I want to take after all.

Practical Clojure/Java interop, CSV parsing with Apache Commons CSV

2018-04-03T00:00:00+10:00

One particularly powerful aspect of Clojure is that it allows you to use Java features without actually writing any Java. And the best way to get my head around the Java interop features is to do a practical example; something like I'd do for work. And as far as examples go, they don't come any more practical than CSV file reading.

Given I'm just learning Clojure, to have any chance of getting the interop code working, I first need to write the Java version. So how would a Java person, which I'm not, parse CSV? Using Apache Commons CSV I suppose. Here's the CSV:

id,greeting
1,Shiver me timbers!
2,Ahoy there!

CSV parsing in Java

Here's the Java code, PiratePhrases.java:

import java.io.FileReader;
import java.io.IOException;
import org.apache.commons.csv.CSVFormat;
import org.apache.commons.csv.CSVParser;
import org.apache.commons.csv.CSVRecord;

public class PiratePhrases {
    public static void main(String[] args) throws IOException {
        FileReader reader = new FileReader("pirate.csv");
        Iterable<CSVRecord> records = CSVFormat.DEFAULT.withFirstRecordAsHeader().parse(reader);
        for (CSVRecord record : records) {
            System.out.println(record.get("greeting"));
        }
    }
}

As I'm running Guix SD, I'll install Java and Commons CSV and compile and run it like this:

$ guix package --install icedtea java-commons-csv
$ javac -cp "${HOME}/.guix-profile/share/java/commons-csv.jar" PiratePhrases.java
$ java -cp "${HOME}/.guix-profile/share/java/commons-csv.jar" PiratePhrases
Shiver me timbers!
Ahoy there!

I've barely written an Java since university, and don't use any statically typed languages at the moment so it was comforting to that javac demanded I handle IOExceptions; even if I did fob it off by adding the exception to the interface.

CSV parsing in Clojure

Any sane Clojure programmer would probably reach for a Clojure native CSV library, like data.csv, but since this is an exercise, I'm going to stick with Commons CSV and translate this Java code as literally as I can. The Clojure version, pirate_phrases.clj, looks like this:

(import java.io.FileReader)
(import org.apache.commons.csv.CSVFormat)
(import org.apache.commons.csv.CSVParser)
(import org.apache.commons.csv.CSVRecord)

(let [reader (FileReader. "pirate.csv")
      records (.parse (.withFirstRecordAsHeader CSVFormat/DEFAULT) reader)]
  (doseq [record records]
    (println (.get record "greeting"))))

And, on Guix SD, is run like this:

guix package --install clojure
java -cp "${HOME}/.guix-profile/share/java/clojure-1.9.0.jar:${HOME}/.guix-profile/share/java/commons-csv.jar" clojure.main pirate_phrases.clj
Shiver me timbers!
Ahoy there!

This was a good chance to get familiar with a few of the interop features:

Importing with (import java.io.FileReader)
Instantiating an object, where new FileReader(path) becomes (FileReader. path)
Accessing a class attribute/method, where CSVFormat.DEFAULT becomes CSVFormat/DEFAULT, and
Calling an instance method, where record.get("greeting") becomes (.get record "greeting").

I particularly like that the Clojure notation clearly distinguishes class attributes/methods from instance methods eg. CSVFormat/DEFAULT vs. (.get record "greeting").

I'll still be reaching for Python when a CSV file comes my way, but I was pleasantly surprised at how neat the Clojure version of this code ended up.

Getting started with the GnuBee Personal Cloud 2 (or 1)

2018-02-01T00:00:00+11:00

Updated 21 September 2018 and 29 August 2019. See bottom of this post.

I just received my GnuBee Personal Cloud 2 in the post. It's a device newly manufactured specifically to run completely free software, making in completely unique. The device is designed for network accessible storage, providing slots for six 3.5" hard drives.

As a brand new product, the documentation for getting started is nearly non-existant though, so here's my journey so far.

Assembly

I was surprised to see that the holes for the circuit board mounting brackets didn't all line up. It turns out that the centre brackets are oriented differently to the end brackets. Look carefully at the photo below.

The GnuBee Personal Cloud 2 (from below)

Getting started

The GBPC comes with the LibreCMC operating system installed in the on-board flash memory.

Based on the Background information for GnuBee PC1, I plugged a network cable in to the black socket and plugged the other end in to my computer. I then tured on the GBPC using the silver switch (red button on GBPC1). No SDCard was installed at this point.

Once it had started up, the computer connected to the GBPC's network. Visiting http://192.168.10.1/ in a web browser showed the LibreCMC login screen. Although it's not made clear, you can initially log in as user "root" by leaving the password blank. You can also connect with SSH with no password by running ssh -o StrictHostKeyChecking root@192.168.10.1 with no password. I needed the -o StrictHostKeyChecking as I also have a GBPC1 that uses the same IP address.

Proceeding to the System/Administration section allows entry of a root password. I set up key-based SSH access at the same time. Leave "Interface" as "unspecified", uncheck password authentication and enter your SSH public key fingerprint (from .ssh/id_rsa.pub). Select "save and apply" at the bottom of the page. I was then able to log out and log back in with my new password, and connect with SSH to root@192.168.10.1.

Unfortunately I notied that after switching the device off and back on again, the password and SSH changes were reset. I'm not sure why.

Resources

The GnuBee devices are not well documented at the moment, but at present the best information can be found at:

Installing hard drives

I installed a hard drive in slot 1 and after resetting the GBPC with the black reset button, noted that the new drive was listed under the System/Mount Points section as filesystem "crytpo_LUKS". The second drive was an unencrypted "ext4" drive. That didn't show up under "Mount Points" automatically and didn't seem to mount when I selected "Add", chose the drive from the list and defined a custom mount point /backup. There was no problem mounting this drive from the SSH command line though.

Further work

At this point I still have a lot of questions, and not enough time to investigate them all. If you know the answers, please get in touch.

Do I need to install a different operating system to do something useful with the device? I was a little suprised to find that the GBPC doesn't do anything much out of the box. Not that this in any way limits the device's potential, but it's nice not to have to do everything yourself. It seems that all drive mounting must be done manually, that there's no file sharing services or even rsync installed by default.
How do you install software on LibreCMC? I tried the System/Software section of the web interface, but installing or searching for packages fails with an error connecting to the package repositories.
How do you boot from the SDCard included in my "deluxe pack"? The card appears to have Debian installed, but after insterting the SDCard and resetting, the device still seem to boot to LibreCMC. Noting that the SDCard is auto-mounted if inserted while LibreCMC is running.
What do the different network ports do? Black is a LAN port. Blue seems to be the WAN port as the device can ping the outside world once plugged in. The yellow port doesn't seem to do anything.
Can drives be made to auto-mount? I noticed that plugging in a USB drive are mounted automatically under /tmp/run/mountd/.
How do you connect with the USB-to-UART cable? I tried plugging it in, connecting with picocom /dev/ttyUSB0 9600 and resetting the GBPC, but only see garbage on the screen. Clearly I need different connection settings.
Why don't password changes stick? Cleary the device is accepting some changes, because if I mkdir /mnt/backup, the directory is still there after a power off and on.
Can it be configured to be a DHCP client, rather than a DCHP server?

Update 5 Feb 2018

I learnt that USB-to-UART works via minicom, eg. sudo minicom -b 57600 -D /dev/ttyUSB0. See the GnuBee documentation. I presume that minicom must use different defaults to picocom.

It appears that a firmware update is required to allow booting from the provided SDCard.

The network interface for LibreCMC can be changed from "static" to "DHCP client".

I'm having trouble mounting some encrypted drives. It appears that missing kernel support for "aes-xts-plain64" may be the cause.

Update 7 Feb 2018

Plugged two RAID1 drives into the GBPC2 and it correctly recognised the three RAID partitions and allocated them to /dev/md125, /dev/md126 and /dev/md127.

The opkg update command, as well as the web interface fail to update the package list with:

Downloading https://librecmc.org/librecmc/downloads/snapshots/v1.4.1/ramips/mt7621/packages/Packages.gz
*** Failed to download the package list from https://librecmc.org/librecmc/downloads/snapshots/v1.4.1/ramips/mt7621/packages/Packages.gz

Downloading https://librecmc.org/librecmc/downloads/snapshots/v1.4.1/packages/mipsel_24kc/base/Packages.gz
*** Failed to download the package list from https://librecmc.org/librecmc/downloads/snapshots/v1.4.1/packages/mipsel_24kc/base/Packages.gz

Downloading https://librecmc.org/librecmc/downloads/snapshots/v1.4.1/packages/mipsel_24kc/packages/Packages.gz
*** Failed to download the package list from https://librecmc.org/librecmc/downloads/snapshots/v1.4.1/packages/mipsel_24kc/packages/Packages.gz

Collected errors:
 * opkg_download: Failed to download https://librecmc.org/librecmc/downloads/snapshots/v1.4.1/ramips/mt7621/packages/Packages.gz, wget returned 5.
 * opkg_download: Failed to download https://librecmc.org/librecmc/downloads/snapshots/v1.4.1/packages/mipsel_24kc/base/Packages.gz, wget returned 5.
 * opkg_download: Failed to download https://librecmc.org/librecmc/downloads/snapshots/v1.4.1/packages/mipsel_24kc/packages/Packages.gz, wget returned 5.

In contrast, my LibreCMC-based router from ThinkPenguin updates and installs packages with no problems, so clearly something just isn't configured correctly.

Password changes do appear to be saved, and persist across reboots and powering off. It appears though that the small black button does a "factory reset", which resets the LibreCMC login credentials.

The hard drive idling plugin available through the web interface works well.

Slot #1 on the GBPC2 is blocked by the two 12V plastic clips. This is a minor design fault, but you can simply bend the four plastic tabs firmly down to fix the problem.

Update 21 Sep 2018

OpenWRT v18.06 was released on 31 July. This release includes support for the GnuBee PC1 and PC2 devices and represents a merge between OpenWRT and LEDE projects. Here's the GBPC2 commit. Today I've installed in on my GBPC2.

Installing OpenWRT

As per the OpenWRT quickstart and the GBPC2 OpenWRT hardware info, the images are under here. The one I needed was called "gnubee_gb-pc2-initramfs-kernel.bin". I verified the sha256sum of the file and the GnuPG signature provided.

I first plugged the serial cable into my GBPC2 and into my computer and ran sudo minicom -b 57600 -D /dev/ttyUSB0. This is invaluable to be able to see what the device is doing.

I plugged my GBPC2 directly into my computer (not a router) using the black ethernet socket and visited the web interface at http://192.168.1.1. After logging in, I went to "System", "Backup/Flash Firmware". Under "Flash new firmware image", I unchecked "Keep settings" and selected the file "gnubee_gb-pc2-initramfs-kernel.bin".

The verification page pops up and prompts you to verify that the displayed hash matches the file you downloaded. I checked this with md5sum openwrt-18.06.1-ramips-mt7621-gnubee_gb-pc2-initramfs-kernel.bin and proceeded with the install. Following along on the serial console, I could see the device reboot into OpenWRT. I found I had to switch off and on to get an IP address and access the new web interface.

The new firmware to work well enough, but doesn't seem to include support for mounting SATA drives out of the box and would't let me install packages. To access the Internet, changed the existing network interface to "DHCP client", hit "apply and save" and plugged it into my router (weirdly there seems to be some sort of auto-revert requiring you to make the change, re-plug the device and log in in under 30 seconds or so, which took me a few shots). By looking up my router's DHCP leases, I could find what the new IP address was to connect to.

I first logged in via SSH and updated the package list with "opkg update" (web interface would work fine too). When I tried to install a package via the web interface or "opkg install" I got an error like this:

# opkg install rsync
Installing rsync (3.1.3-1) to root...
Collected errors:
 * verify_pkg_installable: Only have 0kb available on filesystem /overlay, pkg rsync needs 137
 * opkg_install_cmd: Cannot install package rsync.

I then downloaded the "gnubee_gb-pc1-squashfs-sysupgrade.bin", which I was now able to install with the "Flash new firmware image" feature, presumably because I already had OpenWRT installed. After repeating all the above I was able to log in and opkg install rsync and see that my SATA drive was detected at /dev/sda.

I had some problems getting the drive to mount. Not entirely sure whether it was permissions or a missing package. I installed "mountd" which enabled automounting, but mounting was failing:

# mount -t ext4 /dev/sda /mnt
mount: mounting /dev/sda on /mnt failed: Invalid argument

I installed a few other packages including "e2fsprogs", "mount-utils" and "kmod-fs-ext4". Eventually I realised that mkdir ~/x; mount /dev/sda1 ~/x worked fine, so changed the permissions on /mnt to 777 which seemed to fix that. Weird though, since I was root. Anyhow, it's working, which is good enough for now.

Update 29 Aug 2019

For some time I've been accessing my GnuBee via it's dynamic IP address, since zeroconf was no longer working after installing v18.06. That's downright painful. Today I learn that this can be fixed by installing avahi:

opkg update
opkg install avahi-daemon-service-ssh avahi-daemon-service-http

See the OpenWRT zeroconf documentation.

Configuring Android K-9 Mail from the command line

2017-10-06T00:00:00+11:00

I hate having to re-enter my email settings in a new or reinstalled device. Secure passwords are particularly painful to transcribe via a phone keyboard, but it's not just the basic settings. It's also those extras like signature, disabling notifications and folders that buy you if they're not right. I've set it all up dozens of times before and will no doubt do it many more.

On my laptop and desktop I have a permanent, version controlled copy of my email settings that I can apply automatically. Surely you could do the same on Android/Replicant? I couldn't find any useful information about people doing this with configuration management tools like Puppet, Salt, Ansible or anything else so here's my first attempt:

#!/usr/bin/env bash

set -x

# Set the K-9 incoming and outgoing account details via ADB and SQLite3.
#
# Does not yet *create* the accounts, so you still need to click through the
# accounts wizard and enter dummy information.#
#
# K-9 account settings live in an SQLite3 database and have a unique identifier
# for each account. Records in the database look like:
#
# primkey                                           | value
# ba46a4cf-bde0-4d5d-aa6d-9da12a8367df.transportUri | xxx
#
# The transportUri and storeUri represent the server settings and are base64
# encoded, presumably to obfuscate the password.

PREFERENCES_DATABASE='/data/data/com.fsck.k9/databases/preferences_storage'
ACCOUNT_UUID_PERSONAL='ba46a4cf-bde0-4d5d-aa6d-9da12a8367df'
ACCOUNT_UUID_WORK='83a40e3e-a4ef-4351-a9f2-cd2fb8510654'

# These weird looking URLs came directly from the database (base64 decoded), eg.
# sqlite3 preferences_storage "select * from preferences_storage"
PERSONAL_STOREURI='imap+ssl+://PLAIN:ben%2540stumbles.id.au:password1@imap.fastmail.com:993/1%7C'
PERSONAL_TRANSPORTURI='smtp+ssl+://ben%2540stumbles.id.au:password1:PLAIN@smtp.fastmail.com:465'
WORK_STOREURI='imap+ssl+://PLAIN:ben%2540sturm.com.au:password2@imap.fastmail.com:993/1%7C'
WORK_TRANSPORTURI='smtp+ssl+://ben%2540sturm.com.au:password2:PLAIN@smtp.fastmail.com:465'
WORK_SIGNATURE=$(<~/dotfiles/.signature)

read -r -d '' QUERY <<EOF
    UPDATE preferences_storage SET VALUE = 'Ben Sturmfels' WHERE primkey = '${ACCOUNT_UUID_PERSONAL}.name.0';
    UPDATE preferences_storage SET VALUE = 'Personal' WHERE primkey = '${ACCOUNT_UUID_PERSONAL}.description';
    UPDATE preferences_storage SET VALUE = '$(echo -n $PERSONAL_STOREURI | base64)' WHERE primkey = '${ACCOUNT_UUID_PERSONAL}.storeUri';
    UPDATE preferences_storage SET VALUE = '$(echo -n $PERSONAL_TRANSPORTURI | base64)' WHERE primkey = '${ACCOUNT_UUID_PERSONAL}.transportUri';
    UPDATE preferences_storage SET VALUE = 'false' WHERE primkey = '${ACCOUNT_UUID_PERSONAL}.notifyNewMail';
    UPDATE preferences_storage SET VALUE = '' WHERE primkey = '${ACCOUNT_UUID_PERSONAL}.signature.0';
    UPDATE preferences_storage SET VALUE = 'Archive' WHERE primkey = '${ACCOUNT_UUID_PERSONAL}.sentFolderName';
    UPDATE preferences_storage SET VALUE = 'Junk Mail' WHERE primkey = '${ACCOUNT_UUID_PERSONAL}.spamFolderName';


    UPDATE preferences_storage SET VALUE = 'Ben Sturmfels' WHERE primkey = '${ACCOUNT_UUID_WORK}.name.0';
    UPDATE preferences_storage SET VALUE = 'Work' WHERE primkey = '${ACCOUNT_UUID_WORK}.description';
    UPDATE preferences_storage SET VALUE = '$(echo -n $WORK_STOREURI | base64)' WHERE primkey = '${ACCOUNT_UUID_WORK}.storeUri';
    UPDATE preferences_storage SET VALUE = '$(echo -n $WORK_TRANSPORTURI | base64)' WHERE primkey = '${ACCOUNT_UUID_WORK}.transportUri';
    UPDATE preferences_storage SET VALUE = 'false' WHERE primkey = '${ACCOUNT_UUID_WORK}.notifyNewMail';
    UPDATE preferences_storage SET VALUE = '--
${WORK_SIGNATURE}' WHERE primkey = '${ACCOUNT_UUID_WORK}.signature.0';
    UPDATE preferences_storage SET VALUE = 'Archive' WHERE primkey = '${ACCOUNT_UUID_WORK}.sentFolderName';
EOF

# As my config grew, ADB started telling me "error: service name too long", so I
# had to write the configuration to a shell script, transfer it to the phone and
# run it.
TEMPFILE=$(tempfile)
echo "sqlite3 $PREFERENCES_DATABASE \"$QUERY\"" > $TEMPFILE
adb push $TEMPFILE /data/local/tmp
adb shell sh /data/local/tmp/$(basename ${TEMPFILE})

The idea is that you'd plug in your Android/Replicant phone by USB with ADB debugging enabled and run the above shell script. I gather that you need to kill K-9 mail for the settings to take effect (long hold on primary button and close K-9).

Alright, now I'll freely admit that this took quite a bit more time than it would of to just complete the K-9 settings wizard, but it's the principle that counts! The great thing about Free Software is that I can have confidence that I'll be using K-9 for many years to come, so this investment of time is worthwhile.

I haven't tried creating accounts from scratch. It could be just generating a UUID identifier and setting [UUID].accountNumber to say 2 for a third account. Or there might be some place you need to register the UUID. That's work for another day.

Disabling Django's password strength checking for development

2017-03-05T00:00:00+11:00

Django 1.9 comes with a feature that enforces strong passwords. It's an excellent security feature, but password complexity is not what you want in development. Here's how to turn the feature off to allow simple passwords.

It goes like this. You've created a new database, run the migrations and are about to create a superuser account:

$ python manage.py createsuperuser
Username (leave blank to use 'user'):
Email address: user@example.com
Password:
Password (again):
This password is too short. It must contain at least 8 characters.
Password:
...

Gahhh! All I want to be able to do is use the same one-letter password for all my development environments. The story is similarly frustrating when creating new user accounts via the Django Admin. You're probably creating a test account for a colleague. You want something short and simple.

Django Administration password validation

The solution is to set the following in your development settings. If you're following best practises, will be something like settings/local.py or settings/dev.py:

AUTH_PASSWORD_VALIDATORS = []

Just promise me you won't do this in production!

Python, FTPS and mis-configured servers

2016-12-09T00:00:00+11:00

Today's project involves automatically uploading electrical metering data to an FTPS server (explicit FTP over TLS, otherwise knowns as ESFTP). Shouldn't be a problem, since Python supports FTPS out of the box. Only it doesn't work. Here's the code:

import ftplib
ftp = ftplib.FTP_TLS('host', 'user', 'password')
ftp.set_debuglevel(1)
ftp.cwd('directory')
with open('filename', 'rb') as f:
    ftp.storbinary('STOR filename', f)
ftp.quit()

After a successful initial connection, the data transfer connection fails:

*cmd* 'PASV'
*resp* '227 Entering Passive Mode (10,200,0,100,255,150).'
*cmd* 'CWD collect/CSV'
*resp* '250 CWD command successful'
*cmd* 'TYPE I'
*resp* '200 Type set to I'
*cmd* 'PASV'
*resp* '227 Entering Passive Mode (10,200,0,100,255,123).'

Traceback (most recent call last):
  File "metering/__main__.py", line 143, in main
    ftp.storbinary('STOR {}'.format(filename), stream)
  File ".../lib/python3.5/ftplib.py", line 503, in storbinary
    with self.transfercmd(cmd, rest) as conn:
  File ".../lib/python3.5/ftplib.py", line 398, in transfercmd
    return self.ntransfercmd(cmd, rest)[0]
  File ".../lib/python3.5/ftplib.py", line 793, in ntransfercmd
    conn, size = FTP.ntransfercmd(self, cmd, rest)
  File ".../lib/python3.5/ftplib.py", line 360, in ntransfercmd
    source_address=self.source_address)
  File ".../lib/python3.5/socket.py", line 711, in create_connection
    raise err
  File ".../lib/python3.5/socket.py", line 702, in create_connection
    sock.connect(sa)
OSError: [Errno 113] No route to host

The most confusing aspect was that the transfer worked perfectly well via FileZilla or Gnome "Connect to Server".

I eventually noticed a message in the FileZilla logs, Server sent passive reply with unroutable address. Using server address instead. It turns out that the FTPS server was mis-configured and was replying to the PASV command with an internal IP address that was not accessible from the public internet. It seems that this is a common enough configuration issue that some FTP clients detect the problem and use the existing server address instead. Python's FTP client doesn't do this though.

The solution was to sub-class the FTP_TLS class and force it to ignore the response:

class FTP_TLS_IgnoreHost(ftplib.FTP_TLS):
    def makepasv(self):
        _, port = super().makepasv()
        return self.host, port

ftp = FTP_TLS_IgnoreHost('host', 'user', 'password')

The makepasv method parses the remote server's response to PASV and returns a host and a port. We're extending this method to throw away the returned host and use the one we already have from the original connection. The underscore is just a convention to indicate that we don't care about the first item in the tuple, the host (it's special in some languages like Prolog, but not in Python). Note of course that this approach doesn't attempt to detect unroutable addresses like FileZilla, it just assumes.

There are no doubt third-party packages that provide this functionality and more, but for such a small extension, it wasn't worth the hassle.

Join us for a swim at Brown Hill

2016-12-03T00:00:00+11:00

The water may be a little chilly, but to coincide with today's opening of the Brown Hill outdoor pool for the summer, we're launching the first page of the Brown Hill Community Hub website — a page about the pool.

We've been getting more involved the local community over the past 6 months or so, following the formation of the Brown Hill Partnership. This is a City of Ballarat initiative to involve a wide range of community groups in choosing projects to fund. So far we've seen the publication of the Brown Hill Community Newsletter, the inaugural Brown Hill Community Festival and today, the launch of the Brown Hill Community Hub. Over the next few months we'll be extending the website to include information about local businesses, sporting clubs and community grounds as well as info about the Hall, the Progress Association, the History Project and Fire Aware.

Phwew, that's certainly a lot of "community". But with the weather warming up and the festive season just around the corner, it's an exciting time of the year to be out and about. Don't forget your togs!

Angular eats my default form values

2016-07-26T00:00:00+10:00

Today, AngularJS isn't doing what I expect. I've auto-generating a really long form in Django, and adding ng-model attributes so I can summarise the form. Unfortunately, Angular is overwriting my defaults. Here's a shortened example:

<html>
    <head>
    </head>
    <body>
        <form ng-app="testApp" ng-controller="TestCtrl">
            <select name="colour" ng-model="colour">
                <option value="red">Red</option>
                <option value="green" selected="selected">Green</option>
                <option value="blue">Blue</option>
            </select>
            <p>You favourite colour is <span style="color: {{ colour }}">{{ colour }}</span>.</p>
        </form>
        <script src="https://ajax.googleapis.com/ajax/libs/angularjs/1.5.7/angular.min.js"></script>
        <script>
         angular.module('testApp', [])
                .controller('TestCtrl', function($scope) {
                    $scope.$watch('colour', function(newValue) {
                        console.log('Colour is: ' + newValue );
                    });
                });
        </script>
    </body>
</html>

It looks something like this:

After marvelling at how little code you need to do something useful, I note that although the HTML form has Green "selected", the select box is initially has no value selected. Why is that? The reason is that once Angular has loaded, it sets the <select> field to its own initial value of undefined, overwriting the HTML default value.

Enter: Angular Auto Value

Thankfully, John Wright and Glauco Custódio have both come up with libraries to make Angular do what we expect. These are Angular Auto Value and Angular Initial Value respectively. Here's our code after modification to use Angular Auto Value (extra <script> element and 'auto-value' argument to angular.module):

<html>
    <head>
    </head>
    <body>
        <form ng-app="testApp" ng-controller="TestCtrl">
            <select name="colour" ng-model="colour">
                <option value="red">Red</option>
                <option value="green" selected="selected">Green</option>
                <option value="blue">Blue</option>
            </select>
            <p>You favourite colour is <span style="color: {{ colour }}">{{ colour }}</span>.</p>
        </form>
        <script src="https://ajax.googleapis.com/ajax/libs/angularjs/1.5.7/angular.min.js"></script>
        <!-- Extra script -->
        <script src="https://cdn.jsdelivr.net/angular.auto-value/latest/angular-auto-value.min.js"></script>
        <script>
         angular.module('testApp', ['auto-value']) // Extra 'auto-value' dependency
                .controller('TestCtrl', function($scope) {
                    $scope.$watch('colour', function(newValue) {
                        console.log('Colour is: ' + newValue );
                    });
                });
        </script>
    </body>
</html>

I often reach for a pinch of Angular as a "jQuery substitute". A touch of auto-completion or form validation adds a lot of value to a server-side web app. Like jQuery, there's no assumption that your whole system is written in JavaScript. Unlike jQuery, it remains maintainable as you add more and more features.

Bruising first experience with ClojureScript

2016-05-20T00:00:00+10:00

"Bruised" is a little how I feel about my first experience porting a program to ClojureScript — configuring production builds has a few traps for newcomers. That said, I am very pleased with the results and excited to do more working with ClojureScript.

My Dad is a soil conservation officer with DELWP. Many years ago I wrote a small Java applet for use in his workshops. It help farmers estimate how much water can flow down an earthen channel without causing soil erosion. Here's a screenshot of the original Java version:

Channel flow calculator (Java version)

Despite its simplicity, the channel flow calculator has been a useful and popular tool. The only drawback has been the inconsistent support for Java applets in web browsers; particularly in government departments. To solve this, we recently decided to rewrite it in HTML and JavaScript. It's also a chance to indulge my interest in ClojureScript!

Why ClojureScript?

Well, less JavaScript for a start. Rather than trying to incrementally fix JavaScript's lack of modules, non-existent standard library, unexpected type coercion rules and inconsistent browser support, ClojureScript starts afresh with solid language fundamentals and good design choices based the Clojure Lisp dialect.

ClojureScript compiles down to lowest-common-denominator JavaScript, meaning that my code will run consistently on any modern browser. I can confidently use the full breadth of the elegant language features, powerful data-types and the convenient standard library. I get real module support. Last but not least, thanks to the optimising compiler, I can fearlessly pull in large libraries, knowing that my visitors will only download the code they need.

ClojureScript trap 1: Unwanted `:main`

The basic translation from Java to ClojureScript was straightforward; even the Java and JavaScript canvas-drawing APIs are nearly identical. After some messing around, I even managed to get Lein, Clojure's de-facto build tool, auto-rebuilding and live-updating my web browser (via plugins lein-cljsbuild and lein-figwheel). The trouble came when I needed to produce the optimised production version for publication.

ClosureScript utilises the Google Closure compiler to remove unused JavaScript code, turning around 1MB into 90kB (20kB Gzipped). All this by simply setting :optimizations :advanced in the build settings.

The first trap was that the :main setting used in development builds can't be left in place when you switch on :optimizations :advanced. Not that Clojure gives up this information willingly. Here's my faulty build script:

;; build.clj
(require 'cljs.build.api)

(cljs.build.api/build "src"
  {:main 'channel_flow.core
   :output-to "out/main.js"
   :optimizations :advanced})

(System/exit 0)

And here's the output:

$ java -cp cljs-1.8.51 clojure.main build.clj
Exception in thread "main" java.lang.AssertionError: Assert failed: No file for namespace channel_flow.core exists

Not so helpful. That error kept me busy for nearly two hours, including diving into the ClojureScript source code to figure out what was going on. After removing the :main setting, the error disappears. It's worth noting that if you're building through lein-cljsbuild, it does the right thing and ignores the unnecessary :main setting.

ClojureScript trap 2: Symbol renaming

The second trap was that after switching from :optimizations :simple to :optimizations :advanced, I was seeing JavaScript errors like document.ga is undefined. It's a bizarre feeling when your code was working only moments ago. Optimisation isn't meant to change the behaviour of code, right?

Turns out it does, if you're not abiding by certain rules. I was using the following HTML form:

<form name="mannings">
<label>Top width: <input type="number" name="w1" /></label>
…

I was referencing this in ClojureScript to trigger the recalculation when inputs were changed:

;; Don't do this
(.addEventListener document.mannings "input" redraw)

The Google Closure compiler was helpfully renaming document.mannings to document.ga to make my code shorter; thereby breaking the reference to the HTML form:

// "Optimised" (broken) JavaScript
document.ga.addEventListener("input",Oe);

All this makes sense in hindsight; I just didn't expect that enabling compilation optimisation do anything other than slow down the build and produce a smaller output file.

The solution is to explicitly advise the Google Closure compiler that document.mannings is something external to this file and not be renamed. Create an externs.js file that references the variables:

// externs.js
document.mannings = {
    w1: null,
    w2: null,
    cd: null,
    s: null,
    m: null,
    fd: null,
    v: null,
    q: null
};

Declare the externs to the compiler in your build script:

:optimizations :advanced
:externs ["externs.js"]

The results

Channel flow calculator (HTML/JavaScript version)

After clearing these initial hurdles, the updated channel flow calculator is now working well (picture below). A few weeks on, now that my frustration has faded, I'm actually really looking forward to my next ClojureScript project.

Nginx dynamic image resizing with caching

2016-04-21T00:00:00+10:00

At Sturm we build web apps that perform beautifully on both tiny mobile devices and huge desktop monitors. To achieve these, we use Nginx to resize images on-the-fly. This approach was used in our recent Course Finder project with Federation University.

The Nginx web server comes with a handy image filter module, but most tutorials don't configure it with performance in mind. Performance means caching.

Let's start with the basics:

server {
    # Basic image resizing server, no caching.
    server_name example.com;

    location ~ "^/media/(?<width>\d+)/(?<image>.+)$" {
        alias /var/www/images/$image;
        image_filter resize $width -;
        image_filter_jpeg_quality 75;
        image_filter_buffer 8M;
    }
}

Don't forget to reload your Nginx configuration:

$ sudo service nginx force-reload

This server block will respond to a request like http://example.com/media/768/mountains.jpg with the file /var/www/images/mountains.jpg. It will resize the image down to 768 pixels wide if necessary and compress it using JPEG quality 75. Requests where the original image is 8MB will be rejected.

Unfortunately our simple example doesn't perform so well, since it does a resize for every single request. Let's test it using Apache Bench (from apache2-utils on a Debian-based GNU/Linux). We'll make 100 requests using 5 concurrent connections:

$ ab -n 100 -c 5 http://example.com/media/768/mountains.jpg
...
Requests per second:    4.14 [#/sec] (mean)
...

Oh no, only four requests per second!

The solution is to cache the resized image so that the work is only done once a day. Since Nginx can't cache and resize at the same time, we'll need a trick. That trick is to use two server blocks. The first will be an internal-onlyserver that resize images. The second will be a public server that proxies requests to our internal server and then caches the result:

server {
    # Internal image resizing server.
    server_name localhost;
    listen 8888;

    location ~ "^/media/(?<width>\d+)/(?<image>.+)$" {
        alias /var/www/images/$image;
        image_filter resize $width -;
        image_filter_jpeg_quality 75;
        image_filter_buffer 8M;
    }
}

proxy_cache_path /tmp/nginx-images-cache/ levels=1:2 keys_zone=images:10m inactive=24h max_size=100m;

server {
    # Public-facing cache server.
    server_name example.com;

    # Only serve widths of 768 or 1920 so we can cache effectively.
    location ~ "^/media/(?<width>(768|1920))/(?<image>.+)$" {
        # Proxy to internal image resizing server.
        proxy_pass http://localhost:8888/media/$width/$image;
        proxy_cache images;
        proxy_cache_valid 200 24h;
    }

    location /media {
        # Nginx needs you to manually define DNS resolution when using
        # variables in proxy_pass. Creating this dummy location avoids that.
        # The error is: "no resolver defined to resolve localhost".
        proxy_pass http://localhost:8888/;
    }
}

Now that we're caching the resized images, Apache Bench shows a slow first request, but thereafter serves over 7000 requests per second!

Lots of love for Conservancy at LCA 2016

2016-02-09T00:00:00+11:00

It was wonderful to see an outpouring of support for Conservancy last week at LCA 2016. A number of speakers highlighted Conservancy during their talks, Bradley Kuhn gave a talk about GPL compliance, we held a Supporter lunch (photo below), and both the conference organisers and the new Linux Australia President highlighted Conservancy in the conference closing.

Software Freedom Conservancy is a tiny organisation doing amazing work to support the development of Free Software. They assist community free software projects with their administrative work (financial, legal, etc.) so that the developers can concentrate on making great software. Conservancy's member projects include BusyBox, Git, Inkscape, BusyBox, Mercurial, PyPy, Samba, Twisted and Wine.

Speaking on MediaGoblin at linux.conf.au 2016

2016-02-02T00:00:00+11:00

I'm very excited to be speaking about GNU MediaGoblin this Friday 5 February at linux.conf.au in Geelong, Australia. MediaGoblin is a media publishing tool for artists.

I began using MediaGoblin nearly four years ago as a way to share photos of our new baby with our family and close friends — a popular use case from all accounts! The two compelling features at the time were the ability to upload both images, audio and video to the same service and the automatic resizing, transcoding and compression. Being a technical person, I theoretically had all the tools and knowledge to convert the media and publish the HTML, but in practise it simply wouldn't have happened. With MediaGoblin, the publishing process is a joyful one.

A lot has changed in MediaGoblin since then. GStreamer 1.0 has brought more robust transcoding with support for more formats. The client-to-server API allows publishing media from a pump.api client. There's great new features for admin users including reprocessing of failed transcoding, command-line uploads, bulk uploads, reporting/moderation, PDF/document media type, 3D model media type and much more. These advances have allowed me to also use MediaGoblin to publish audio, video and slides from my past presentations on Emacs, Python, free software and software patents.

Thanks to Jessica Tallon's work, funded entirely by your donations, the project is also looking forward to announcing federation in the near future. This will bring social features such as sharing, commenting and notifications across different distinct MediaGoblin servers. I can't wait to talk more about these exciting developments on Friday. I'm very grateful to Chris Webber and Jessica for walking me through some of the details of their current work.

Hope to see you at 10:40am in D4.303 Costa Theatre!

Grunt build hangs on ngtemplates:dist

2015-10-28T00:00:00+11:00

I just ran into a strange situation where grunt build on a Yeoman Angular project was hanging at the following step:

$ grunt build
⋮
Running "ngtemplates:dist" (ngtemplates) task

The cause turned out to be an extra double quote symbol in one of the HTML view templates:

<form method="get" action="#" onsubmit="window.location='#/results/' + query.value; return false;"">

A git bisect helped reveal the problematic revision, but even so it took a lot of trial and error to find the cause. The challenge was that Grunt gave no clues as to what might be causing it to stall. It also didn't crash, rather sat there in an infinite loop merrily consuming the CPU.

Interesting, the problem doesn't occur for just any old extra quote, so I'd guess that means that Angular is trying to parse this snippet of JavaScript and getting stuck. And yes, I agree that this is a very un-Angular way to submit a form. :)

Auto-starting Emacs smerge-mode for Git

2015-09-28T00:00:00+10:00

I love that Emacs notices merge conflicts in Bazaar versioned files, highlighting the differences in green and pink. It's a small thing, but I really miss this automation for when working with Git — I have to manually type M-x smerge-mode.

The feature works by looking for conflict files and markers whenever you open a Bazaar versioned file:

;;; Copied from Emacs 24.3.1: vc/vc-bzr.el.gz:472
(defun vc-bzr-find-file-hook ()
  (when (and buffer-file-name
             ;; FIXME: We should check that "bzr status" says "conflict".
             (file-exists-p (concat buffer-file-name ".BASE"))
             (file-exists-p (concat buffer-file-name ".OTHER"))
             (file-exists-p (concat buffer-file-name ".THIS"))
             ;; If "bzr status" says there's a conflict but there are no
             ;; conflict markers, it's not clear what we should do.
             (save-excursion
               (goto-char (point-min))
               (re-search-forward "^<<<<<<< " nil t)))
    ;; TODO: the merge algorithm used in `bzr merge' is nicely configurable,
    ;; but the one in `bzr pull' isn't, so it would be good to provide an
    ;; elisp function to remerge from the .BASE/OTHER/THIS files.
    (smerge-start-session)
    (add-hook 'after-save-hook 'vc-bzr-resolve-when-done nil t)
    (message "There are unresolved conflicts in this file")))

The above looks a little complicated, but the important part is the line (smerge-start-session). This starts smerge-mode to highlight merge conflicts.

Let's do something similar with Git by defining a function vc-git-find-file-hook:

(defun vc-git-find-file-hook ()
  (when (save-excursion
      (goto-char (point-min))
      (re-search-forward "^<<<<<<< " nil t))
    (smerge-start-session)))

Emacs vc-mode has a standard interface for version control systems, so simply defining this function is all we need to do. Emacs will look for and run the function if it exists (it doesn't by default).

Similar to the Bazaar version, this function looks for conflict marker text in the file, and starts smerge-mode if it finds any. The Git version is shorter because Git deals with conflicts differently, and also because I haven't hit any edge cases… yet.

John Oliver on Software Patents

2015-06-05T00:00:00+10:00

A few years back, only the legal and computing industry understood the dangers of software patents. Today I was amazed watch John Oliver's detailed and hilarious treatment of the subject on Last Week Tonight (YouTube). The narrator summed up our concerns beautifully:

In the last decade, the number of software patents has skyrocketed. The issue here, say experts, is that while patents for machines tend to be fairly specific, software patents can be so broad and vague that they may give someone the ability to later claim ownership for inventions they never dreamed of at the time.

I suppose it's tempting for reporters to dwell on patent trolls, but many of the most harmful patents come from large well-known companies and consortia. These patents make important fields such as video encoding almost unbearable to work in and leave the public all the poorer for lack of compatibly and choice. That aside, it's always satisfying to see Uniloc being berated for it's misuse of the patent system.

Rock Your Emacs video from LibrePlanet

2015-04-18T00:00:00+10:00

I've recently returned from LibrePlanet 2015 where I presented a workshop called Rock Your Emacs. The workshop is designed to help Emacs users over the initial barriers to writing their own customisations with Lisp code. While we're waiting for official videos to be published, here's the video, slides and exercises from my workshop.

Rock Your Emacs, LibrePlanet 2015

Slides (PDF)
Exercises (Emacs Lisp)

Do you love Emacs, but have never understood the strange code with lots of brackets? You're missing out on one of the great joys of Emacs — customizing it to work exactly the way you want. It turns out that Emacs is a full Lisp code interpreter, so once you know a little Emacs Lisp, almost anything is possible.

After attending this tutorial, you will know how to:

read basic Emacs Lisp code
modify Emacs Lisp code as well as writing your own
make persistent customizations to your Emacs
bind your favourite commands to keys
answer your own questions with the amazing help system
use built-in Emacs Lisp development tools like the debugger

Programming experience is helpful but not required. Bring your Emacs and type along.

Ben Sturmfels is a software engineer and free software activist from Ballarat, Australia. He organises Free Software Melbourne, leads the End Software Patents Australia campaign, rides bikes, flies kites and grows vegetables.

Controlling fan speed on Thinkpad X60

2014-04-09T00:00:00+10:00

There’s just one minor annoyance with my new LibreBoot Thinkpad X60 from GLUGLUG. Working it hard causes it to overheat and shut down. Thankfully, the fan turns out to have much more capability than the defaults allow; all but solving this problem.

In Trisquel GNU/Linux 6.0, the X60′s automatic fan controller runs at one of eight preset speeds (0-7), depending on the temperature. About 10 mins with both CPUs running at 100% will cause it to overheat (eg. two copies of python -c"while True: pass").

As described on ThinkWiki, you can manually override the fan control by adding the line options thinkpad_acpi fan_control=1 to /etc/modprobe.d/thinkpad_acpi.conf. Then after rebooting and as root, you can set the fan to run at maximum speed (~3700 RPM) with:

echo level 7 > /proc/acpi/ibm/fan

Only that’s not the true maximum speed. You can actually disable the rotation speed regulation and just supply full voltage to the fan (~4900 RPM):

echo level full-speed > /proc/acpi/ibm/fan

This setting is a bit noisier, but allows both CPUs to run at 100% with no signs of overheating. A good result.

Next I need to figure out how figure out how to enable this automatically when the machine gets hot. I’ve tried thinkfan, but it seems to only allow the default 0-7 speeds used by the default fan control.

Raising concerns about ACTA and TPP

2012-07-14T00:00:00+10:00

The recent resounding defeat of ACTA in the European Parliament has given me a lot of hope. Many citizens took the time to contact members of parliament and this seems to have made all the difference. I hope that Australian can also reject this harmful agreement. That won’t happen without some effort though.

This morning I had the opportunity to meet and speak with Australian Government MP Catherine King. I raised concerns about the ACTA and TPP agreements. She acknowledged my concerns and agreed to contact Minister Stephen Conroy about these issues. Here are the points I provided:

ACTA

Please defend our freedom by voting down this harmful agreement.

Negotiated in secret for over two years
Uses the guise of reducing commercial international-level copyright and trademark infringement (an acceptable aim) to introduce provisions that restrict the freedom of citizens
Serves the interest of large incumbent music and movie companies, preserving their outdated business models
Punishes Internet users with disconnection if accused of sharing
Prohibits software that circumvents digital-restrictions on text/audio/video (ie. “you must only use our locked-down e-book reader software to read this book”)
Encourages a culture of surveillance and suspicion in which freedom and creativity are seen as dangerous.
Following a huge public backlash, the European Parliament recently voted ACTA down, 478 votes to 39.

TPP

Please speak out against the nature and scope of this agreement.

Completely non-transparent process – NGOs could make submissions at stakeholder forums knowing almost nothing about the scope of the agreement
Leaked draft indicates that United States is using the TPP to spread it’s draconian copyright and patent laws around the world.
Expected to increase the duration and scope of copyright and patents and adding harsher infringement penalties. These laws are already a long way in the favour of publishing and media companies to the detriment of authors, artists and consumers.
Expected to explicitly include areas of computation and information processing in patentable subject matter (aka. software patents). These patents don’t work as a way to encourage innovation, actually stifling the Australian software industry (see our 2010 petition which received 1000 signatures).

Update 21 July 2012: Looks like I wasn’t paying attention. Australia signed up to ACTA in October 2011. The TPP is definitely still being negotiated though.

Choice, please speak out against DRM

2010-01-08T00:00:00+11:00

*The following is a letter written to Choice, the leading Australian consumer advocacy organisation.

I urge Choice to speak out against products like the Amazon Kindle electronic book reader. They use Digital Restrictions Management (DRM) technology to restrict your basic freedoms.

The Kindle uses DRM technology to give Amazon full control over what the customer can and can’t do with the product. It allows many freedoms of traditional books to be stripped off, such as sharing the book with friend or giving the book away when you’re finished with it. DRM also allows monitoring and censoring of what you read. For example, Amazon recently remotely deleted copies of George Orwell’s “Animal Farm” and “1984″. The customers involved were given no notice or choice. The breathtakingly irony is that “1984″ describes a fictional society where citizens are under constant surveillance and control.

Understandably, devices like the Kindle are widely considered to be unethical. DRM is also being added to many products like phones, music players, digital music, films and books. The result is we consumers lose. Please help defend our rights by considering these issues in future reviews and campaigns.

More information about DRM is available at the Defective By Design website.

Stumbles is Ben Sturmfels

Disabling the “HP-Setup” wifi on an HP LaserJet

Talks I've enjoyed (August 2022)

Tim Ewald - Clojure: Programming with Hand Tools (2013)

How to fix uWSGI "OSError: write error"

Problem

How to reproduce

Solution

Getting started with Guix deploy

Overall strategy

A web server from scratch

Further work

Wrong turns I took along the way

Thanks and further information

This week I learnt (Week 33, 2021)

SVG supports SMIL animation — no JavaScript or CSS

GitHub commit squashing — no need to rebase

Reordering commits during git rebase

Talks I've watched (March 2021)

Emily Ashley: 7 Falsehoods Programmers Believe about Place & Time

Baishampayan Ghose: Revenge of the Pragmatists

Why MediaGoblin?

MyGovID ethical, privacy and security problems for ATO Business Portal

GitLab CI/CD on OpenShift Online Kubernetes… or not

An aside: How is this meant to work?

Practical Clojure/Java interop, CSV parsing with Apache Commons CSV

CSV parsing in Java

CSV parsing in Clojure

Getting started with the GnuBee Personal Cloud 2 (or 1)

Assembly

Getting started

Resources

Installing hard drives

Further work

Update 5 Feb 2018

Update 7 Feb 2018

Update 21 Sep 2018

Installing OpenWRT

Update 29 Aug 2019

Configuring Android K-9 Mail from the command line

Disabling Django's password strength checking for development

Python, FTPS and mis-configured servers

Join us for a swim at Brown Hill

Angular eats my default form values

Enter: Angular Auto Value

Bruising first experience with ClojureScript

Why ClojureScript?

ClojureScript trap 1: Unwanted :main

ClojureScript trap 2: Symbol renaming

The results

Nginx dynamic image resizing with caching

Lots of love for Conservancy at LCA 2016

Speaking on MediaGoblin at linux.conf.au 2016

Grunt build hangs on ngtemplates:dist

Auto-starting Emacs smerge-mode for Git

John Oliver on Software Patents

Rock Your Emacs video from LibrePlanet

Rock Your Emacs, LibrePlanet 2015

Controlling fan speed on Thinkpad X60

Raising concerns about ACTA and TPP

ACTA

TPP

Choice, please speak out against DRM

ClojureScript trap 1: Unwanted `:main`