news aggregator

Stewart Smith: Fedora 25 + Lenovo X1 Carbon 4th Gen + OneLink+ Dock

Planet Linux Australia - Mon, 2017-05-29 13:00

As of May 29th 2017, if you want to do something crazy like use *both* ports of the OneLink+ dock to use monitors that aren’t 640×480 (but aren’t 4k), you’re going to need a 4.11 kernel, as everything else (for example 4.10.17, which is the latest in Fedora 25 at time of writing) will end you in a world of horrible, horrible pain.

To install, run this:

sudo dnf install \ \ \ \ \ \ \ \

This grabs a kernel that’s sitting in testing and isn’t yet in the main repositories. However, I can now see things on monitors, rather than 0 to 1 monitor (most often 0). You can also dock/undock and everything doesn’t crash in a pile of fail.

I remember a time when you could fairly reliably buy Intel hardware and have it “just work” with the latest distros. It’s unfortunate that this is no longer the case, and it’s more of a case of “wait six months and you’ll still have problems”.


(at least Wayland and X were bug for bug compatible?)

Michael Still: Configuring docker to use rexray and Ceph for persistent storage

Planet Linux Australia - Mon, 2017-05-29 13:00
For various reasons I wanted to play with docker containers backed by persistent Ceph storage. rexray seemed like the way to do that, so here are my notes on getting that working...

First off, I needed to install rexray:

    root@labosa:~/rexray# curl -sSL | sh Selecting previously unselected package rexray. (Reading database ... 177547 files and directories currently installed.) Preparing to unpack rexray_0.9.0-1_amd64.deb ... Unpacking rexray (0.9.0-1) ... Setting up rexray (0.9.0-1) ... rexray has been installed to /usr/bin/rexray REX-Ray ------- Binary: /usr/bin/rexray Flavor: client+agent+controller SemVer: 0.9.0 OsArch: Linux-x86_64 Branch: v0.9.0 Commit: 2a7458dd90a79c673463e14094377baf9fc8695e Formed: Thu, 04 May 2017 07:38:11 AEST libStorage ---------- SemVer: 0.6.0 OsArch: Linux-x86_64 Branch: v0.9.0 Commit: fa055d6da595602715bdfd5541b4aa6d4dcbcbd9 Formed: Thu, 04 May 2017 07:36:11 AEST

Which is of course horrid. What that script seems to have done is install a deb'd version of rexray based on an alien'd package:

    root@labosa:~/rexray# dpkg -s rexray Package: rexray Status: install ok installed Priority: extra Section: alien Installed-Size: 36140 Maintainer: Travis CI User <travis@testing-gce-7fbf00fc-f7cd-4e37-a584-810c64fdeeb1> Architecture: amd64 Version: 0.9.0-1 Depends: libc6 (>= 2.3.2) Description: Tool for managing remote & local storage. A guest based storage introspection tool that allows local visibility and management from cloud and storage platforms. . (Converted from a rpm package by alien version 8.86.)

If I was building anything more than a test environment I think I'd want to do a better job of installing rexray than this, so you've been warned.

Next to configure rexray to use Ceph. The configuration details are cunningly hidden in the libstorage docs, and aren't mentioned at all in the rexray docs, so you probably want to take a look at the libstorage docs on ceph. First off, we need to install the ceph tools, and copy the ceph authentication information from the the ceph we installed using openstack-ansible earlier.

    root@labosa:/etc# apt-get install ceph-common root@labosa:/etc# scp -rp . The authenticity of host ' (' can't be established. ECDSA key fingerprint is SHA256:SA6U2fuXyVbsVJIoCEHL+qlQ3xEIda/MDOnHOZbgtnE. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '' (ECDSA) to the list of known hosts. rbdmap 100% 92 0.1KB/s 00:00 ceph.conf 100% 681 0.7KB/s 00:00 ceph.client.admin.keyring 100% 63 0.1KB/s 00:00 ceph.client.glance.keyring 100% 64 0.1KB/s 00:00 ceph.client.cinder.keyring 100% 64 0.1KB/s 00:00 ceph.client.cinder-backup.keyring 71 0.1KB/s 00:00 root@labosa:/etc# modprobe rbd

You also need to configure rexray. My first attempt looked like this:

    root@labosa:/var/log# cat /etc/rexray/config.yml libstorage: service: ceph

And the rexray output sure made it look like it worked...

    root@labosa:/etc# rexray service start ● rexray.service - rexray Loaded: loaded (/etc/systemd/system/rexray.service; enabled; vendor preset: enabled) Active: active (running) since Mon 2017-05-29 10:14:07 AEST; 33ms ago Main PID: 477423 (rexray) Tasks: 5 Memory: 1.5M CPU: 9ms CGroup: /system.slice/rexray.service └─477423 /usr/bin/rexray start -f May 29 10:14:07 labosa systemd[1]: Started rexray.

Which looked good, but /var/log/syslog said:

    May 29 10:14:08 labosa rexray[477423]: REX-Ray May 29 10:14:08 labosa rexray[477423]: ------- May 29 10:14:08 labosa rexray[477423]: Binary: /usr/bin/rexray May 29 10:14:08 labosa rexray[477423]: Flavor: client+agent+controller May 29 10:14:08 labosa rexray[477423]: SemVer: 0.9.0 May 29 10:14:08 labosa rexray[477423]: OsArch: Linux-x86_64 May 29 10:14:08 labosa rexray[477423]: Branch: v0.9.0 May 29 10:14:08 labosa rexray[477423]: Commit: 2a7458dd90a79c673463e14094377baf9fc8695e May 29 10:14:08 labosa rexray[477423]: Formed: Thu, 04 May 2017 07:38:11 AEST May 29 10:14:08 labosa rexray[477423]: libStorage May 29 10:14:08 labosa rexray[477423]: ---------- May 29 10:14:08 labosa rexray[477423]: SemVer: 0.6.0 May 29 10:14:08 labosa rexray[477423]: OsArch: Linux-x86_64 May 29 10:14:08 labosa rexray[477423]: Branch: v0.9.0 May 29 10:14:08 labosa rexray[477423]: Commit: fa055d6da595602715bdfd5541b4aa6d4dcbcbd9 May 29 10:14:08 labosa rexray[477423]: Formed: Thu, 04 May 2017 07:36:11 AEST May 29 10:14:08 labosa rexray[477423]: time="2017-05-29T10:14:08+10:00" level=error msg="error starting libStorage server" error.driver=ceph time=1496016848215 May 29 10:14:08 labosa rexray[477423]: time="2017-05-29T10:14:08+10:00" level=error msg="default module(s) failed to initialize" error.driver=ceph time=1496016848216 May 29 10:14:08 labosa rexray[477423]: time="2017-05-29T10:14:08+10:00" level=error msg="daemon failed to initialize" error.driver=ceph time=1496016848216 May 29 10:14:08 labosa rexray[477423]: time="2017-05-29T10:14:08+10:00" level=error msg="error starting rex-ray" error.driver=ceph time=1496016848216

That's because the service is called rbd it seems. So, the config file ended up looking like this:

    root@labosa:/var/log# cat /etc/rexray/config.yml libstorage: service: rbd rbd: defaultPool: rbd

Now to install docker:

    root@labosa:/var/log# sudo apt-get update root@labosa:/var/log# sudo apt-get install linux-image-extra-$(uname -r) \ linux-image-extra-virtual root@labosa:/var/log# sudo apt-get install apt-transport-https \ ca-certificates curl software-properties-common root@labosa:/var/log# curl -fsSL | sudo apt-key add - root@labosa:/var/log# sudo add-apt-repository \ "deb [arch=amd64] \ $(lsb_release -cs) \ stable" root@labosa:/var/log# sudo apt-get update root@labosa:/var/log# sudo apt-get install docker-ce

Now let's make a rexray volume.

    root@labosa:/var/log# rexray volume ls ID Name Status Size root@labosa:/var/log# docker volume create --driver=rexray --name=mysql \ --opt=size=1 A size of 1 here means 1gb mysql root@labosa:/var/log# rexray volume ls ID Name Status Size rbd.mysql mysql available 1

Let's start the container.

    root@labosa:/var/log# docker run --name some-mysql --volume-driver=rexray \ -v mysql:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql Unable to find image 'mysql:latest' locally latest: Pulling from library/mysql 10a267c67f42: Pull complete c2dcc7bb2a88: Pull complete 17e7a0445698: Pull complete 9a61839a176f: Pull complete a1033d2f1825: Pull complete 0d6792140dcc: Pull complete cd3adf03d6e6: Pull complete d79d216fd92b: Pull complete b3c25bdeb4f4: Pull complete 02556e8f331f: Pull complete 4bed508a9e77: Pull complete Digest: sha256:2f4b1900c0ee53f344564db8d85733bd8d70b0a78cd00e6d92dc107224fc84a5 Status: Downloaded newer image for mysql:latest ccc251e6322dac504e978f4b95b3787517500de61eb251017cc0b7fd878c190b

And now to prove that persistence works and that there's nothing up my sleeve...
    root@labosa:/var/log# docker run -it --link some-mysql:mysql --rm mysql \ sh -c 'exec mysql -h"$MYSQL_PORT_3306_TCP_ADDR" \ -P"$MYSQL_PORT_3306_TCP_PORT" -uroot -p"$MYSQL_ENV_MYSQL_ROOT_PASSWORD"' mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 3 Server version: 5.7.18 MySQL Community Server (GPL) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | +--------------------+ 4 rows in set (0.00 sec) mysql> create database demo; Query OK, 1 row affected (0.03 sec) mysql> use demo; Database changed mysql> create table foo(val char(5)); Query OK, 0 rows affected (0.14 sec) mysql> insert into foo(val) values ('a'), ('b'), ('c'); Query OK, 3 rows affected (0.08 sec) Records: 3 Duplicates: 0 Warnings: 0 mysql> select * from foo; +------+ | val | +------+ | a | | b | | c | +------+ 3 rows in set (0.00 sec)

Now let's re-create the container and prove the data remains.

    root@labosa:/var/log# docker stop some-mysql some-mysql root@labosa:/var/log# docker rm some-mysql some-mysql root@labosa:/var/log# docker run --name some-mysql --volume-driver=rexray \ -v mysql:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql 99a7ccae1ad1865eb1bcc8c757251903dd2f1ac7d3ce4e365b5cdf94f539fe05 root@labosa:/var/log# docker run -it --link some-mysql:mysql --rm mysql \ sh -c 'exec mysql -h"$MYSQL_PORT_3306_TCP_ADDR" -\ P"$MYSQL_PORT_3306_TCP_PORT" -uroot -p"$MYSQL_ENV_MYSQL_ROOT_PASSWORD"' mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 3 Server version: 5.7.18 MySQL Community Server (GPL) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> use demo; Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> select * from foo; +------+ | val | +------+ | a | | b | | c | +------+ 3 rows in set (0.00 sec)
So there you go.

Tags for this post: docker ceph rbd rexray
Related posts: So you want to setup a Ceph dev environment using OSA; Juno nova mid-cycle meetup summary: containers


David Rowe: LilacSat-1 Codec 2 in Space!

Planet Linux Australia - Mon, 2017-05-29 11:03

On May 25th LilacSat-1 was launched from the ISS. The exiting news is that it contains an analog FM to Codec 2 repeater. I’ve been in touch with Wei Mingchuan, BG2BHC during the development phase, and it’s wonderful to see the satellite in orbit. He reports that some Hams have had preliminary contacts.

The LilacSat-1 team have developed their own waveform, that uses a convolutional code running over BPSK at 9600 bit/s. Wei reports a MDS of about -127 dBm on a USRP B210 SDR which is quite respectable and much better than analog FM. GNU radio modules are available to support reception. I think it’s great that Wei and team have used open source (including Codec 2) to develop their own novel systems, in this case a hybrid FM/digital system with custom FEC and modulation.

Now I need to get organised with some local hams and find out how to work this satellite myself!


The LilacSat-1 page has links to GNU Radio modules that can be used to receive signals from the satellite.

OpenSTEM: This Week in HASS – term 2, week 7

Planet Linux Australia - Mon, 2017-05-29 09:03

This week students are starting to round off their main body of assessable work for the term. Older students are completing and starting to present their presentations, while younger students have posters and models to finish off.

Foundation/Prep/Kindy to Year 3

Students in our stand-alone Foundation/Prep/Kindy unit (F.2) are continuing to explore with their senses this week. While still working on their model or collage for their Favourite Place, they are using their sense of Smell to consider which aromas they like or dislike. Teachers (and students) can bring in a range of things with different smells to explore in class. Ideas for these are given in the Teacher’s Handbook. An important part of this investigation is considering how one can represent one’s favourite smells in the model or collage – students might try to draw the objects associated with the smells, or see if they can find creative alternatives to represent this sense.

Students in integrated Foundation/Prep/Kindy and Year 1 classes (Unit F.6) and those in Years 1 (Unit 1.2), 2 (Unit 2.2) and 3 (Unit 3.2) are completing their posters to be displayed next week. These posters cover topics of local significance – either local history information, or dealing with issues, such as littering or the need for play equipment. As the students work on the posters, teachers are holding discussions with them on responsibility for different issues. The delegation of responsibility to members of the community, local government, other authorities, people who use the facilities, the school P&C, the students etc. should be discussed in class, so that students start to understand how people have different responsibilities in different situations. The teacher can also revisit issues of responsibility in the classroom – what are the students responsible for? What is the teacher responsible for? What is the school responsible for? These discussions are an important means of allowing the students to interact and practise group discussion skills, as well as helping them to think about responsibilities.

Years 3 to 6

Students in Year 3 (Unit 3.6) are completing their presentation on an extreme climate explorer and may start presenting it to the class this week. Year 4 students (Unit 4.2) are presenting on their explorer of Africa and South America. Year 5 students (Unit 5.2) are presenting on their chosen explorer from North America; and Year 6 students (Unit 6.2) are presenting on their chosen explorer from Asia. The remaining 3 weeks of this unit are allocated to the presentations, to ensure enough time for these to be given in full. The presentations should cover all the aspects raised over previous weeks and answered in the Student Workbook – the environments and geography of the areas explored; sustainability issues, such as extinction of animals and changes in local environments; characteristics of the countries involved in the explorations; reasons for explorations and how these created the background which led to the settlement of Australia and the role of indigenous people, as well as impact on indigenous people and their environments. The presentation is thus a comprehensive body of work.


David Rowe: FreeDV 2017 Road Map

Planet Linux Australia - Mon, 2017-05-29 09:03

Half through the year but I thought I better write down some plans anyway! Helps me organise my thoughts and minimise the tangential work. The main goal for 2017 is a FreeDV mode that is competitive with SSB at low SNRs on HF channels. But first, lets see what happened in 2016….

Achievements in 2016

Here is the 2016 Roadmap. Reviewing it, we actually made good progress on a bunch of the planned activities:

  • Brady O’Brien (KC9TPA) worked with me to develop the FreeDV 2400A and 2400B modes.
  • Fine progress on the SM2000 project, summarised nicely in my Gippstech 2016 SM2000 talk. Thanks in particular to Brady, Rick (KA8BMA) and Neil (VK5KA).
  • The open telemetry work provided some key components for the Wenet system for high speed SSDV images from High Altitude Balloons. This work spun out of FSK modem development by Brady and myself, combined with powerful LDPC FEC codes from VK5DSP, with lots of work by Mark VK5QI, and AREG club members. It operates close to the limits of physics: with a 100mW signals we transmit HD images over 100km at 100 kbit/s using a $20 SDR and a good LNA. Many AREG members have set up Wenet receive stations using $100 roadkill laptops refurbished with Linux. It leaves commercial telemetry chips-sets in the dust, about 10dB behind us in terms of performance.
  • Ongoing FreeDV outreach via AREG FreeDV broadcasts, attending conferences and Hamfests. Thank you to all those who promote and use FreeDV.

FreeDV 2017 Roadmap

Codec 2 700C is the breakthrough I have been waiting for. Communications quality, conversational speech at just 700 bit/s, and even on a rough first pass it outperforms MELPe at 600 bit/s. Having a viable codec at 700 bit/s lets us consider powerful LDPC FEC codes in the 2000Hz SSB type bandwidths I’m targeting, which has led to a new OFDM modem and the emerging FreeDV 700D mode. I now feel comfortable that I can reach the goal of sub zero dB SNR digital speech that exceeds SSB in quality.

So here is the 2017 roadmap. Partially shaded work packages are partially complete. The pink work packages are ongoing activities rather than project based:

Rather than push FreeDV 700D straight out, I have decided to have another iteration at Codec 2 quality, using the Codec 700C algorithms as a starting point. The FreeDV 700D work has suggested we can use latency to overcome the HF channel, which means frames of several hundred ms to several seconds. By exploring correlation over longer Codec 2 frames we can achieve lower bit rates (e.g. sub 400 bit/s) or get better voice quality at 700 bit/s and above.

I’ve been knocking myself out to get good results at low SNRs. However many HF and indeed VHF/UHF PTT radio conversions take place at SNRs of greater than 10dB. This allows us to support higher bit rate codecs, and achieve better speech quality. For example moving from 0dB to 10dB means 10 times the bit rate at the same Bit Error Rate (BER). The OFDM modem will allow us to pack up to 4000 bit/s into a 2000 Hz SSB channel.

The algorithms that work so well for Codec 2 700C can be used to increase the quality at higher bit rates. So the goal of the “Codec 2 Quality” work package is to (i) improve the quality at 700-ish bit/s, and (ii) come up with a Codec 2 mode that improves on the speech quality of Codec 2 1300 (as used in the FreeDV 1600 mode) at 2000-ish bit/s.

After the Codec 2 quality improvement I will port the new algorithms to C, release and tune on the FreeDV GUI program, then port to the SM1000. Fortunately the new OFDM modem is simpler in terms of memory and computation that the COHPSK modem used on FreeDV 700C. We have an option to use a short LDPC code (224 bits) which I feel that will run OK on the SM1000 with a little work.

Putting it all Together

The outputs will be a low SNR mode competitive with SSB at low SNR, and (hopefully) a high SNR mode that sounds better than SSB at medium to high SNRs. It will be available as a free software download (FreeDV GUI program), an embedded stand-alone product (the SM1000), and as a gcc library (FreeDV API)

Then we can get back to VHF/UHF work and the SM2000 project.

Help Wanted

When will this all happen? Much sooner if you help!

I’m a busy guy, making steady progress in the field of open source digital radio. While I appreciate your ideas, and enjoy brainstorming as much as the next person, what I really want are your patches, and consistent week in/week out effort. If you can code in C and/or have/are willing to learn a little GNU Octave, there is plenty of work to be done in SM1000 maintenance, a port of the OFDM modem to C, and SM1000 hardware/software maintenance, and FreeDV GUI program refactoring and maintenance. Email me.


FreeDV 2016 Roadmap. Promises, promises……
FreeDV 2400A and 2400B modes.
SM2000 – First post introducing the SM2000 project
Gippstech 2016 SM2000 talk – Good summary of SM2000 project to date.
LowSNR site – from Bill (VK5DSP) modem guru and LDPC code-smith.
Horus 39 – Fantastic High Speed SSDV Images – Good summary of Wenet blog posts and modem technology.
SM1000 Digital Voice Adaptor.
AREG FreeDV broadcasts.

Michael Still: So you want to setup a Ceph dev environment using OSA

Planet Linux Australia - Sun, 2017-05-28 13:00
Support for installing and configuring Ceph was added to openstack-ansible in Ocata, so now that I have a need for a Ceph development environment it seems logical that I would build it by building an openstack-ansible Ocata AIO. There were a few gotchas there, so I want to explain the process I used.

First off, Ceph is enabled in an openstack-ansible AIO using a thing I've never seen before called a "Scenario". Basically this means that you need to export an environment variable called "SCENARIO" before running the AIO install. Something like this will do the trick?L:

    export SCENARIO=ceph

Next you need to set the global pg_num in the ceph role or the install will fail. I did that with this patch:

    --- /etc/ansible/roles/ceph.ceph-common/defaults/main.yml 2017-05-26 08:55:07.803635173 +1000 +++ /etc/ansible/roles/ceph.ceph-common/defaults/main.yml 2017-05-26 08:58:30.417019878 +1000 @@ -338,7 +338,9 @@ # foo: 1234 # bar: 5678 # -ceph_conf_overrides: {} +ceph_conf_overrides: + global: + osd_pool_default_pg_num: 8 ############# @@ -373,4 +375,4 @@ # Set this to true to enable File access via NFS. Requires an MDS role. nfs_file_gw: true # Set this to true to enable Object access via NFS. Requires an RGW role. -nfs_obj_gw: false \ No newline at end of file +nfs_obj_gw: false

That of course needs to be done after the Ceph role has been fetched, but before it is executed, so in other words after the AIO bootstrap, but before the install.

And that was about it (although of course that took a fair while to work out). I have this automated in my little install helper thing, so I'll never need to think about it again which is nice.

Once Ceph is installed, you interact with it via the monitor container, not the utility container, which is a bit odd. That said, all you really need is the Ceph config file and the Ceph utilities, so you could move those elsewhere.

    root@labosa:/etc/openstack_deploy# lxc-attach -n aio1_ceph-mon_container-a3d8b8b1 root@aio1-ceph-mon-container-a3d8b8b1:/# ceph -s cluster 24424319-b5e9-49d2-a57a-6087ab7f45bd health HEALTH_OK monmap e1: 1 mons at {aio1-ceph-mon-container-a3d8b8b1=} election epoch 3, quorum 0 aio1-ceph-mon-container-a3d8b8b1 osdmap e20: 3 osds: 3 up, 3 in flags sortbitwise,require_jewel_osds pgmap v36: 40 pgs, 5 pools, 0 bytes data, 0 objects 102156 kB used, 3070 GB / 3070 GB avail 40 active+clean root@aio1-ceph-mon-container-a3d8b8b1:/# ceph osd tree ID WEIGHT TYPE NAME UP/DOWN REWEIGHT PRIMARY-AFFINITY -1 2.99817 root default -2 2.99817 host labosa 0 0.99939 osd.0 up 1.00000 1.00000 1 0.99939 osd.1 up 1.00000 1.00000 2 0.99939 osd.2 up 1.00000 1.00000

Tags for this post: openstack osa ceph openstack-ansible
Related posts: Configuring docker to use rexray and Ceph for persistent storage


OpenSTEM: This Week in HASS, term 2 week 5

Planet Linux Australia - Thu, 2017-05-25 17:04

NAPLAN’s over and it’s time to sink our teeth into the main body of curriculum work before mid-year reporting rolls around. Our younger students are using all their senses to study the environment and local area around them, whilst our older students are hard at work on their Explorer projects.

Foundation/Prep/Kindy to Year 3

Unit F.2 for stand-alone Foundation/Prep/Kindy classes has the students continuing to think about their Favourite Place. This week students are considering what they can hear in their Favourite Place and how they will depict that in their model of their Favourite Place. Students can also think about what their Favourite Sounds are and whether or not these would occur in their Favourite Place. Students in integrated Foundation/Prep/Kindy classes (Unit F.6) and Years 1 (Unit 1.2), 2 (Unit 2.2) and 3 (Unit 3.2) have this week set aside for an excursion to a local park or area of heritage significance. If an excursion outside school grounds is impractical teachers can achieve similar results from an excursion around the school and oval. Students are using their senses to interpret their environment, as well as thinking about living and non-living things, natural and managed landscapes and sources of heat and light.

Years 3 to 6

Students in Years 3 (Unit 3.6), 4 (Unit 4.2), 5 (Unit 5.2) and 6 (Unit 6.2) are continuing their project on an explorer. This week the focus for most students is on animals which may have been encountered by their explorer. Year 3 students are examining animals from different climate zones and how they are adapted to deal with climate extremes. Students in Years 4 and 5 look at extinct animals from Africa, South America or North America, assessing impact and sustainability issues. Students in Year 4 (and optionally as an extension for Year 3) consider the life cycle of their chosen animal. Students in years 4, 5 and 6 also start to examine the differences between Primary and Secondary sources and some of the OpenSTEM resources contain quotes or copies of primary material, so that students can refer to these in their project. Year 6 students are examining the changing Economies and Politics of Asia through time, in order to place the explorations within a broader context and to gain a greater understanding of the development of the global situation. Students have another 2 weeks to complete their presentation on their explorer (including environment and other aspects), before assessment of this project.

David Rowe: Urban HF Noise

Planet Linux Australia - Thu, 2017-05-25 13:03

Over the past 30 years, HF radio noise in urban areas has steadily increased. S6-S9 noise levels are common, which makes it hard to listen to the signals we want to receive.

I’ve been wondering if we can attenuate this noise using knowledge of the properties of the noise, and some clever DSP. Even 6dB would be useful, that’s like the transmitting station increasing their power by a factor of 4. I’ve just spent 2 months working on a 4dB improvement in my FreeDV work. So this week I’ve been messing about with pen and paper and a few simulations, exploring the problem of man-made noise on HF radio.

PWM Noise

One source of noise is switching power supplies, which have short, high current pulses flowing through them at a rate of a few hundred kHz. A series of short impulses in the time domain produces a series of spectral lines (i.e sinusoids or tones) in the frequency domain, so a 200kHz switcher produces tones at 200kHz, 400kHz, 600kHz etc. These tones are the “birdies” we hear as we tune our HF radios. The shorter the pulses are, the higher in frequency they will extend.

Short pulses lead to efficient switch mode power supplies, which is useful for energy efficiency, and especially desirable for high power devices like electric car chargers and solar panel inverters. So the trend is shorter switching times, higher currents and therefore more HF noise.

The power supplies adjust the PWM pulse-width back and forth as they adjust to varying conditions, which introduces a noise component. This is similar to phase noise in oscillators, and causes a continuous noise floor to appear in addition to the tones. The birdies we can tune around, but the noise floor sets a limit on urban HF operations.

The Octave script impulse_noise.m was used to generate the plots in this post. Here is a plot of some PWM impulse samples (top), and the HF spectrum.

I’ve injected a “wanted” signal at 1MHz for comparison. Given a switcher frequency of 255kHz, with 0.1V impulse amplitude, the noise floor is -90dBV down, or about 10uV. This is S5-S6 level noise, assuming 0.1V impulse amplitude induced onto our antenna by local switcher noise, e.g. nearby house wiring, or the neighbors TV. These numbers seem reasonable and match what we hear in our receivers.

Single Pulses

Single, isolated pulses are an easier problem. Examples are lightning or man-made sources that produce pulses at a rate slower than the bandwidth of the signal we are interested in.

A single impulse produces a flat spectrum, so the noise at frequency f Hz is almost the same as the noise at frequency f+delta Hz, where delta is small. This means you can use the noise at frequencies next to the one you are interested in to estimate and remove the noise in your frequency of interest.

Here is an impulse that lasts two samples, the magnitude spectrum changes slowly, although the phase changes quickly due to the time offset of the impulse.

Turns out that if the impulse position is known, and most of the energy is confined to that impulse, we can make a reasonable estimate of the noise at one frequency, from the noise at adjacent frequencies. Below we estimate the phase and magnitude (green cross) of frequency bin H(k+1) (nearby blue cross) from bin H(k). I’ve actually plotted H(k-1), H(k), and H(k+1) for comparison. The error in the estimation is -44dB down, so that’s a lot of noise removed.

Unfortunately this gets harder when there are multiple impulses in the same time window, and I can’t work out how to remove noise is this case. However this idea might be useful for some classes of impulse noise.

Noise Blanker

Another idea I tried was “blanking” out the impulses, buy opening and closing a switch so that the impulses are not allowed into the receiver. This works OK when we have a wideband signal, but falls over when just a bandpass version is available. In the bandpass version the “pulse” is smeared over time and we are no longer able to gate it out.

There will also be problems dealing with multiple PWM signals, that have different timing and frequency.

I haven’t looked at samples of the RF received from any real world switcher signals yet. I anticipate the magnitude and phase of the switcher signal will be all over the place, due to some torturous transfer function between the switcher and the terminals of my receiver. Plus various other signals will be present. Possibly there is a wide spectrum (short noise pulses) that we can work with. However I’d much rather deal with narrow bandpass signals consisting of just our wanted signal plus the switcher noise floor.

Next Steps

I might get back to my FreeDV work now, and leave this work on the back burner. I do feel I’m getting my head around the problem, and developing a “bag of tricks” that will be useful when other pieces fall into place.

The urban noise appears to be localised, e.g. if you head out into the country the background noise level is much lower. This suggests it’s coupled into the HF antenna by some local effect like induction. So another approach is to estimate the noise using a separate receiver that just picks up the local noise, through a sense antenna that is inefficient for long distance HF signals.

The local noise sequence could then be subtracted from the HF signal. I am aware of analog boxes that do this, using a magnitude and phase network to match the differences in signals received by the sense and HF antennas.

However a DSP approach will allow a more complex relationship (like an impulse response that extends for several microseconds) between the two antenna signals, and allow automatic adjustment. The noise spectrum can change quickly, as PWM is modulated and multiple devices turn on and off in the neighborhood. However the relationship between the two antennas will change slowly if they are fixed in space. This problem reminds me of echo cancellation, something I have played with before. Given radio hardware is now very cheap ($20 SDR dongles), multiple receivers could also be used.

So my gut feel remains that HF urban noise can be reduced to some extent (e.g. 6 or 12dB suppression) using DSP. If those nasty PWM switchers are inducing RF voltages into our antennas, we can work out a way to subtract those voltages.

Danielle Madeley: Announcing new high-level PKCS#11 HSM support for Python

Planet Linux Australia - Mon, 2017-05-22 23:02

Recently I’ve been working on a project that makes use of Thales HSM devices to encrypt/decrypt data. There’s a number of ways to talk to the HSM, but the most straight-forward from Linux is via PKCS#11. There were a number of attempts to wrap the PKCS#11 spec for Python, based on SWIG, cffi, etc., but they were all (a) low level, (b) not very Pythonic, (c) have terrible error handling, (d) broken, (e) inefficient for large files and (f) very difficult to fix.

Anyway, given that nearly all documentation on how to actually use PKCS#11 has to be discerned from C examples and thus I’d developed a pretty good working knowledge of the C API, and I’ve wanted to learn Cython for a while, I decided I’d write a new binding based on a high level wrapper I’d put into my app. It’s designed to be accessible, pick sane defaults for you, use generators where appropriate to reduce work, stream large files, be introspectable in your programming environment and be easy to read and extend.

It’s currently a work in progress, but it’s now available on pip. You can get a session on a device, create a symmetric key, find objects, encrypt and decrypt data. The Cryptoki spec is quite large, so I’m focusing on the support that I need first, but it should be pretty straightforward for anyone who wanted to add something else they needed. I like to think I write reasonably clear, self-documenting code.

At the moment it’s only tested on SoftHSMv2 and the Thales nCipher Edge, which is what I have access to. If someone at Amazon wanted this to work flawlessly on CloudHSM, send me an account and I’ll do it Then I can look at releasing my Django integrations for fields, storage, signing, etc.

OpenSTEM: This Week in HASS – term 2, week 6

Planet Linux Australia - Mon, 2017-05-22 09:03

This week students doing the Understanding Our World™ program are exploring their environment and considering indigenous peoples. Younger students are learning about local history and planning a poster on a local issue. Older students are studying indigenous peoples around the world. All the students are working strongly on their main pieces of assessment for the term.

Foundation/Prep/Kindy to Year 3

Our youngest students, using the stand-along Foundation/Prep/Kindy unit (F.2) are exploring the sense of touch in their environment this week. Students consider a range of fabrics and textiles and choose which ones match their favourite place, for inclusion in their model or collage. Students in integrated classes of Foundation/Prep/Kindy and Year 1 (Unit F.6), Year 1 students (Unit 1.2), Year 2 students (Unit 2.2) and Year 3 (Unit 3.2) are starting to prepare a poster on an issue regarding their school, or local park/heritage place, while considering the local history. These investigations should be based on the excursion from last week. Students will have 2 weeks to prepare their posters, for display either at the school or a local venue, such as the library or community hall.

Years 3 to 6

Students in Years 3 to 6 are continuing with their project on an explorer. Students in Year 3 (Unit 3.6) are examining Australian Aboriginal groups from extreme climate areas of Australia, such as the central deserts, or cold climate areas. Students then choose one of these groups to describe in their Student Workbook, and add to their presentation. Students in Year 4 (Unit 4.2) are studying indigenous peoples of Africa and South America. They will then select a group from the area visited by their explorer, to include in their presentation. Year 5 students (Unit 5.2) do the same with indigenous groups from North America; whilst year 6 students (Unit 6.2) have a wide range of resources on indigenous peoples from Asia to select for study and inclusion in their presentation. Resources are available on groups from across mainland Asia (such as the Mongols, Tatars, Rus, Han), as well as South-East Asia (such as Malay, Dyak, Dani etc.). This is the last section of work to be included in the presentation, and students will then finish their presentation and present it to the class.

OpenSTEM: Borrowing a Pencil

Planet Linux Australia - Fri, 2017-05-19 13:04

Student: Can I borrow a pencil?

Teacher: I don’t know. Can you?

Student: Yes. I might add that colloquial irregularities occur frequently in any language. Since you and the rest of our present company understood perfectly my intended meaning, being particular about the distinctions between “can” and “may” is purely pedantic and arguably pretentious.

Teacher: True, colloquialism and the judicious interpretation of context help us communicate with nuance, range, and efficiency. And yet, as your teacher, my job is to teach you to think about language with care and rigour. Understanding the shades of difference between one word and another, and to think carefully about what you want to say, will give you greater power and versatility in your speech and writing.

Student: Point taken. May I have a pencil?

Teacher: No, you may not. We do not have pencils since the department cut funding for education again last year.

Danielle Madeley: PostgreSQL date ranges in Django forms

Planet Linux Australia - Fri, 2017-05-19 13:01

Django’s postgres extensions support data types like DateRange which is super useful when you want to query your database against dates, however they have no form field to expose this into HTML.

Handily Django 1.11 has made it super easy to write custom widgets with complex HTML.

Start with a form field based off MultiValueField:

from django import forms from psycopg2.extras import DateRange class DateRangeField(forms.MultiValueField): """ A date range """ widget = DateRangeWidget def __init__(self, **kwargs): fields = ( forms.DateField(required=True), forms.DateField(required=True), ) super().__init__(fields, **kwargs) def compress(self, values): try: lower, upper = values return DateRange(lower=lower, upper=upper, bounds='[]') except ValueError: return None

The other side of a form field is a Widget:

from django import forms from psycopg2.extras import DateRange class DateRangeWidget(forms.MultiWidget): """Date range widget.""" template_name = 'forms/widgets/daterange.html' def __init__(self, **kwargs): widgets = ( forms.DateInput(), forms.DateInput(), ) super().__init__(widgets, **kwargs) def decompress(self, value): if isinstance(value, DateRange): return (value.lower, value.upper) elif value is None: return (None, None) else: return value class Media: css = { 'all': ('//',) # noqa: E501 } js = ( '//', '//', '//', # noqa: E501 )

Finally we can write a template to use the jquery-date-range-picker:

{% for widget in widget.subwidgets %} <input type="hidden" name="{{ }}"{% if widget.value != None %} value="{{ widget.value }}"{% endif %}{% include "django/forms/widgets/attrs.html" %} /> {% endfor %} <div id='container_for_{{ }}'></div>

With a script block:

(function() { var format = 'D/M/YYYY'; var isoFormat = 'YYYY-MM-DD'; var startInput = $('#{{ }}'); var endInput = $('#{{ }}'); $('#{{ }}').dateRangePicker({ inline: true, container: '#container_for_{{ }}', alwaysOpen: true, format: format, separator: ' ', getValue: function() { if (!startInput.val() || !endInput.val()) { return ''; } var start = moment(startInput.val(), isoFormat); var end = moment(endInput.val(), isoFormat); return start.format(format) + ' ' + end.format(format); }, setValue: function(s, start, end) { start = moment(start, format); end = moment(end, format); startInput.val(start.format(isoFormat)); endInput.val(end.format(isoFormat)); } }); })();

You can now use this DateRangeField in a form, retrieve it from cleaned_data for database queries or store it in a model DateRangeField.

David Rowe: FreeDV 700D – First Over The Air Tests

Planet Linux Australia - Thu, 2017-05-18 21:03

OK so after several attempts I finally managed to push a 700D signal from my QTH in Adelaide (PF95gc) 1170km to the Manly Warringah Radio Society WebSDR in Sydney (QF56oh). Bumped my power up a little, raised my antenna, and hunted around until I found a relatively birdie-free frequency, as even low level birdies are stronger than my very weak signal.

Have a listen:

Analog SSB 700D modem Decoded 700D DV

Here is a spectrogram (i.e. a waterfall with the water falling from left to right) of the analog then digital signal:

Faint birdies (tones) can be seen as horizontal lines at 1000 and 2000 Hz. You can see the slow fading on the digital signal as it dips beneath the noise every few seconds.

The scatter diagram looks like bugs (bits?) splattered on a windscreen:

The slow fading causes the errors to bounce up and down over time (above). The packet error rate (measured on the 28 bit Codec 2 frames) is 26%. This is rather high, but I would argue we have intelligible speech here, and that the intelligibility is better than SSB.


I used 4 interleaver frames, which is about 640ms. Perhaps a longer interleaver would ride over the fades.

I’m impressed! Conditions were pretty bad on 40m, the band was “closed”. This is day 1 of FreeDV 700D. It will improve from here.

Command Lines

The Octave demodulator doing it’s thing:

octave:56> ofdm_rx("~/Desktop/700d_part2/manly5_4.wav",4, "manly5_4.err") Coded BER: 0.0206 Tbits: 12544 Terrs: 259 PER: 0.2612 Tpacketerrs: 117 Tpackets: 448 Raw BER..: 0.0381 Tbits: 26432 Terrs: 1007

Not sure if I’m working out raw and coded BER right as they are not usually this close. Will look into that. Maybe all the errors are in the fades, where both the demod and LDPC decoder fall in a heap.

The ofdm_tx/ofdm_rx system transmits test frames of known data, so we can work out the BER. By xor-ing the tx and rx bits we can generate an error pattern that can be used to insert errors into a Codec 2 700C bit stream, using this magic incantation:

~/codec2-dev/build_linux/src$ sox ~/Desktop/cq_freedv_8k.wav ~/Desktop/cq_freedv_8k.wav -t raw -r 8000 -s -2 - | ./c2enc 700C - - | ./insert_errors - - ../../octave/manly5_4.err 28 | ./c2dec 700C - - | sox -t raw -r 8000 -s -2 - ~/Desktop/manly5_4_ldpc224_4.wav

It’s just like the real thing. Trust me. And it gives me a feel for how the system is hanging together earlier rather than after months more development.


Lots of links on the Towards FreeDV 700D post earlier today.

Michael Still: The Collapsing Empire

Planet Linux Australia - Thu, 2017-05-18 15:00

ISBN: 076538888X
This is a fun fast read, as is everything by Mr Scalzi. The basic premise here is that of a set of interdependent colonies that are about to lose their ability to trade with each other, and are therefore doomed. Oh, except they don't know that and are busy having petty trade wars instead. It isn't a super intellectual read, but it is fun and does leave me wanting to know what happens to the empire...

Tags for this post: book john_scalzi
Related posts: The Last Colony ; The End of All Things; Zoe's Tale; Agent to the Stars; Redshirts; Fuzzy Nation Comment Recommend a book

David Rowe: Towards FreeDV 700D

Planet Linux Australia - Thu, 2017-05-18 11:03

For the last two months I have been beavering away at FreeDV 700D, as part my eternal quest to show SSB who’s house it is.

This work was inspired by Bill, VK5DSP, who kindly developed some short LDPC codes for me; and suggested I could improve on the synchronisation overhead of the cohpsk modem. As an aside – Bill is part if the communications payload team for the QB50 SUSat Cubesat – currently parked at the ISS awaiting launch! Very Kerbal.

Anyhoo – I’ve developed a new OFDM modem that has less syncronisation overhead, works better, and occupies less RF bandwidth (1000 Hz) than the cohpsk modem used for 700C. I have wrapped my head around such arcane mysteries as coding gain and now have LDPC codes playing nicely over that nasty old HF channel.

It looks like FreeDV 700D has a gain of 4dB over 700C. This means error free operation at -2dB SNR for AWGN, and 2dB SNR over a challenging fast fading HF channel (two paths, 1Hz Doppler, 1ms delay).

Major Innovations:

  1. An OFDM modem with with low overhead (small Eb/No penalty) synchronisation, even on fading channels.
  2. Use of LDPC codes.
  3. Long (several seconds) interleaver.
  4. Ruthlessly hunting down any dB’s leaking out of my performance curves.

One nasty surprise was that after a closer look at the short (224,112) LDPC codes, I discovered they don’t give any real improvement over the simple diversity scheme used for FreeDV 700C. However with long interleaving (several seconds) of the short codes, or a long (few thousand bit/several seconds) LDPC code we get an additional 3dB gain. The interleaver allows us to ride over the ups and downs of the fast fading channel.

Interleaving has a few downsides. One is delay, the other is when they fail you lose a big chunk of data.

I’ve avoided delay until now, using the argument that low delay is essential for PTT radio. However I’d like to test long delays and see what the trade off/end user experience is. Once someone is speaking – i.e in the middle of an “over” – I suspect we won’t notice the delay. However it could get confusing in fast handovers. This is experimental radio, designed for very low SNRs, so lets give it a try.

We could send the uncoded data without interleaving – allowing low delay decoding when the SNR is high. A switch could control LDPC decoding, allowing a user selection of coded-high-delay or uncoded-low-delay, like a noise banker. Mark, VK5QI, has suggested interleaver depth also be adjustable which I think is a good idea. The decoder could automagically determine interleaver depth by attempting decoding over a range of depths (1,2,4,8,16 frames etc) and noting when the LDPC code converges.

Or maybe we could use a small, low delay, interleaver, and just live with the fades (like we do on SSB) and get the vocoder to mute or interpolate over them, and enjoy low or modest latency.

I’m also interested to see how the LDPC code mops up errors like static bursts and other real-world HF rubbish that SSB subjects us to even on high SNR channels.

So, lots of room for experimentation. At this stage it’s all in GNU Octave simulation form, no C implementation or FreeDV GUI mode exists yet.

Lots more I could write about the engineering behind the modem, but lets leave it there for now and take a look at some results.


Here is a rather busy set of BER versus SNR curves (click for larger version, and here is an EPS file version):

The 10-2 line is where the codec gets easy to listen to.

Observe far-right green (700C) to black (700D candidate with lots of interleaving) HF curves, which are about 4dB apart. Also the far-left cyan shows 700D working at -3dB SNR on AWGN channels. One dB later (-2dB) LDPC magic stomps all errors.

Here are some speech/modem tone samples on simulated channels:

AWGN -2dB SNR Analog SSB 700D modem 700D DV HF +0.8dB SNR Analog SSB 700D modem 700D DV

The analog samples have a 300 to 2600 Hz BPF applied at the tx and rx side, to model an analog SSB radio. The analog SSB and 700D modem signals have exactly the same RMS power and channel models applied to them. In the AWGN channel, it’s difficult to hear the 700D modem signal, however the SSB is audible as it has peaks 9dB above the average.

OK so the 700 bit/s vocoder (Codec 2 700C) speech quality is not great even with no errors, but we have found it supports conversations just fine, and there is plenty of room for improvement. The same techniques (OFDM modem, LDPC interleaving) can also be applied to high quality/high bit rate/high SNR voice modes. But first – I want to push this low SNR DV work through to completion.

Simulation Code

This list summarises the GNU Octave code I’ve developed, as I’ll probably forget the details when I move onto the next project. Feel free to try any of these scripts and let me know what I’ve forgotten to check in. It’s all checked into codec2-dev/octave.

ldpc.m Wrapper functions for using the CML library LDPC functions with Octave ldpcut.m Unit test/demo for ldpc.m ldpc_qpsk.m Runs simulations for a bunch of codes for AWGN and HF channels using a simulated QPSK OFDM modem. Runs at the Rs (the symbol rate), assumes ideal modem ldpc_short.m Simulation used for initial short LDPC code investigation using an ideal rate Rs BPSK modem. Bunch of codes and interleaving schemes tested ofdm_lib.m Library of OFDM modem functions ofdm_rs.m Rate Rs OFDM modem simulation used to develop low overhead pilot symbol phase estimation scheme ofmd_dev.m Rate Fs OFDM modem simulation. This is the real deal, with timing and frequency offset estimation, LDPC integration, and tests for coarse timing and frequency offset estimation ofdm_tx.m Generates test frames of OFDM raw file samples to play over your HF radio ofdm_rx.m Receives raw file samples from your HF radio and 700D-demodulates-decodes, and measures BER and PER

Sing Along

Just this morning I tried to radiate some FreeDV 700D from my home to some interstate SDRs on 40M, but alas conditions were against me. I did manage to radiate across my bench so I know the waveform does make it through real HF radios OK.

Please try sending these files through your radio:

ssb_otx_224_32.wav 32 frame (5.12 second) interleaver ssb_otx_224_4.wav 4 frame (0.64 second) interleaver

Get someone (or a websdr) to sample the received signal (8000Hz sample rate, 16 bit mono), and email me the received file.

Or you can decode it yourself using:

octave:10> ofdm_rx('~/Desktop/otx_224_32_mysample.wav',32);


octave:10> ofdm_rx('~/Desktop/otx_224_4_mysample.wav',4);

The rx side is still a bit rough, I’ll refine it as I try the system with real off-air signals and flush out the bugs.

Update: FreeDV 700D – First Over The Air Tests.


QB50 SUSat cubesat – Bill and team’s Cubesat currently parked at the ISS!
Codec 2 700C and Short LDPC Codes
Testing FreeDV 700C
Modems for HF Digital Voice Part 1
Modems for HF Digital Voice Part 2
FreeDV 700D – First Over The Air Tests

Linux Users of Victoria (LUV) Announce: LUV Beginners May Meeting: Dealing with Security as a Linux Desktop User

Planet Linux Australia - Tue, 2017-05-16 17:02
Start: May 20 2017 12:30 End: May 20 2017 16:30 Start: May 20 2017 12:30 End: May 20 2017 16:30 Location:  Infoxchange, 33 Elizabeth St. Richmond Link:

This presentation will introduce the various aspects of IT security that Linux Desktop users may be grappling with on an ongoing basis. The target audience of the talk will be beginners (newbies) - who might have had bad experiences using Windows OS all these years, and don't know what to expect when tiptoeing into the new world of Linux.  General Linux users who don't always pay much attention to aspects of security may also find interest in sharing some of the commonsense practices that are essential to using our computers safely.

The meeting will be held at Infoxchange, 33 Elizabeth St. Richmond 3121 (enter via the garage on Jonas St.) Late arrivals, please call (0421) 775 358 for access to the venue.

LUV would like to acknowledge Infoxchange for the venue.

Linux Users of Victoria Inc., is an incorporated association, registration number A0040056C.

May 20, 2017 - 12:30

read more

Francois Marier: Recovering from an unbootable Ubuntu encrypted LVM root partition

Planet Linux Australia - Tue, 2017-05-16 14:22

A laptop that was installed using the default Ubuntu 16.10 (xenial) full-disk encryption option stopped booting after receiving a kernel update somewhere on the way to Ubuntu 17.04 (zesty).

After showing the boot screen for about 30 seconds, a busybox shell pops up:

BusyBox v.1.21.1 (Ubuntu 1:1.21.1-1ubuntu1) built-in shell (ash) Enter 'help' for list of built-in commands. (initramfs)

Typing exit will display more information about the failure before bringing us back to the same busybox shell:

Gave up waiting for root device. Common problems: - Boot args (cat /proc/cmdline) - Check rootdelay= (did the system wait long enough?) - Check root= (did the system wait for the right device?) - Missing modules (cat /proc/modules; ls /dev) ALERT! /dev/mapper/ubuntu--vg-root does not exist. Dropping to a shell! BusyBox v.1.21.1 (Ubuntu 1:1.21.1-1ubuntu1) built-in shell (ash) Enter 'help' for list of built-in commands. (initramfs)

which now complains that the /dev/mapper/ubuntu--vg-root root partition (which uses LUKS and LVM) cannot be found.

There is some comprehensive advice out there but it didn't quite work for me. This is how I ended up resolving the problem.

Boot using a USB installation disk

First, create bootable USB disk using the latest Ubuntu installer:

  1. Download an desktop image.
  2. Copy the ISO directly on the USB stick (overwriting it in the process):

    dd if=ubuntu.iso of=/dev/sdc1

and boot the system using that USB stick (hold the option key during boot on Apple hardware).

Mount the encrypted partition

Assuming a drive which is partitioned this way:

  • /dev/sda1: EFI partition
  • /dev/sda2: unencrypted boot partition
  • /dev/sda3: encrypted LVM partition

Open a terminal and mount the required partitions:

cryptsetup luksOpen /dev/sda3 sda3_crypt vgchange -ay mount /dev/mapper/ubuntu--vg-root /mnt mount /dev/sda2 /mnt/boot mount -t proc proc /mnt/proc mount -o bind /dev /mnt/dev


  • When running cryptsetup luksOpen, you must use the same name as the one that is in /etc/crypttab on the root parition (sda3_crypt in this example).

  • All of these partitions must be present (including /proc and /dev) for the initramfs scripts to do all of their work. If you see errors or warnings, you must resolve them.

Regenerate the initramfs on the boot partition

Then "enter" the root partition using:

chroot /mnt

and make sure that the lvm2 package is installed:

apt install lvm2

before regenerating the initramfs for all of the installed kernels:

update-initramfs -c -k all

Michael Still: Python3 venvs for people who are old and grumpy

Planet Linux Australia - Fri, 2017-05-12 15:00
I've been using virtualenvwrapper to make venvs for python2 for probably six or so years. I know it, and understand it. Now some bad man (hi Ramon!) is making me do python3, and virtualenvwrapper just isn't a thing over there as best as I can tell.

So how do I make a venv? Its really not too bad...

First, install the dependencies:

    git clone git:// .pyenv echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bashrc echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bashrc echo 'eval "$(pyenv init -)"' >> ~/.bashrc git clone ~/.pyenv/plugins/pyenv-virtualenv source ~/.bashrc

Now to make a venv, do something like this (in this case, infrasot is the name of the venv):

    mkdir -p ~/.virtualenvs/pyenv-infrasot cd ~/.virtualenvs/pyenv-infrasot pyenv virtualenv system infrasot

You can see your installed venvs like this:

    $ pyenv versions * system (set by /home/user/.pyenv/version) infrasot

Where system is the system installed python, and not a venv. To activate and deactivate the venv, do this:

    $ pyenv activate infrasot $ ... stuff you're doing ... $ pvenv deactivate

I'll probably write wrappers at some point so that this looks like virtualenvwrapper, but its good enough for now.

Tags for this post: python venv virtualenvwrapper python3
Related posts: Implementing SCP with paramiko; Packet capture in python; A pythonic example of recording metrics about ephemeral scripts with prometheus; mbot: new hotness in Google Talk bots; Calculating a SSH host key with paramiko; Twisted conch


OpenSTEM: Assessments

Planet Linux Australia - Fri, 2017-05-12 09:03

Well, NAPLAN is behind us for another year and so we can all concentrate on curriculum work again! This year we have updated our assessment material to make it even easier to map the answers in the Student Workbooks with the curriculum codes. Remember, our units integrate across several curriculum areas. The model answers now contain colour coded curriculum codes that look like this:  These numbers refer to specific curriculum strands, which are now also listed in our Assessment Guides. In the back of each Assessment Guide is a colour coded table – Gold for History; Green for Geography; Light Green for HASS Skills; Orange for Civics and Citizenship; Purple for Economics and Business and Blue for Science. Each curriculum code is included in this table, along with the rubric for grades A to E, or AP to BA for the younger students.

These updates mean that teachers can now match each question to the specific curriculum area being assessed, thus simplifying the process for grading, and reporting on, each curriculum area. So, if you need to report separate grades for Science and HASS, or even History and Civics and Citizenship, you can tally the results across the questions which address those subject areas, to obtain an overall grade for each subject. Since this can be done on a question-by-question basis, you can even keep a running tally of how each student is doing in each subject area through the term, by assessing those questions they have answered, on a regular basis.

Please make sure that you have the latest updates of both the model answers and the assessment guides for each unit that you are teaching, with the codes as shown here. If you don’t have the latest updates, please download them from our site. Log in with your account, go to your downloads (click on “My Account” on the top right and then “Downloads” on the left). Find the Model Answers PDF for your unit(s) in the list of available downloads and click the button(s) to download each one again. Email us if there are any issues.

Paul Wayper: How to get Fedora working on a System 76 Oryx Pro

Planet Linux Australia - Thu, 2017-05-11 21:02
Problems: a) No sound b) Only onboard screen, does not recognise HDMI or Mini-DP Solutions: 1) Install Korora 2) Make sure you're not using an outdated kernel that doesn't have the snd-hda-intel driver available. 3) dnf install akmod-nvidia xorg-x11-drv-nvidia Extra resources: