Planet Linux Australia

Syndicate content
Planet Linux Australia - http://planet.linux.org.au
Updated: 8 min 27 sec ago

Michael Still: How are we going with Nova Kilo specs after our review day?

Mon, 2014-12-15 10:28
Time for another summary I think, because announcing the review day seems to have caused a rush of new specs to be filed (which wasn't really my intention, but hey). We did approve a fair few specs on the review day, so I think overall it was a success. Here's an updated summary of the state of play:







API







API (EC2)



  • Expand support for volume filtering in the EC2 API: review 104450.
  • Implement tags for volumes and snapshots with the EC2 API: review 126553 (fast tracked, approved).




Administrative



  • Actively hunt for orphan instances and remove them: review 137996 (abandoned); review 138627.
  • Check that a service isn't running before deleting it: review 131633.
  • Enable the nova metadata cache to be a shared resource to improve the hit rate: review 126705 (abandoned).
  • Implement a daemon version of rootwrap: review 105404.
  • Log request id mappings: review 132819 (fast tracked).
  • Monitor the health of hypervisor hosts: review 137768.
  • Remove the assumption that there is a single endpoint for services that nova talks to: review 132623.




Block Storage



  • Allow direct access to LVM volumes if supported by Cinder: review 127318.
  • Cache data from volumes on local disk: review 138292 (abandoned); review 138619.
  • Enhance iSCSI volume multipath support: review 134299.
  • Failover to alternative iSCSI portals on login failure: review 137468.
  • Give additional info in BDM when source type is "blank": review 140133.
  • Implement support for a DRBD driver for Cinder block device access: review 134153.
  • Refactor ISCSIDriver to support other iSCSI transports besides TCP: review 130721 (approved).
  • StorPool volume attachment support: review 115716.
  • Support Cinder Volume Multi-attach: review 139580 (approved).
  • Support iSCSI live migration for different iSCSI target: review 132323 (approved).




Cells







Containers Service







Database







Hypervisor: Docker







Hypervisor: FreeBSD



  • Implement support for FreeBSD networking in nova-network: review 127827.




Hypervisor: Hyper-V







Hypervisor: Ironic







Hypervisor: VMWare



  • Add ephemeral disk support to the VMware driver: review 126527 (fast tracked, approved).
  • Add support for the HTML5 console: review 127283.
  • Allow Nova to access a VMWare image store over NFS: review 126866.
  • Enable administrators and tenants to take advantage of backend storage policies: review 126547 (fast tracked, approved).
  • Enable the mapping of raw cinder devices to instances: review 128697.
  • Implement vSAN support: review 128600 (fast tracked, approved).
  • Support multiple disks inside a single OVA file: review 128691.
  • Support the OVA image format: review 127054 (fast tracked, approved).




Hypervisor: libvirt







Instance features







Internal



  • A lock-free quota implementation: review 135296.
  • Automate the documentation of the virtual machine state transition graph: review 94835.
  • Fake Libvirt driver for simulating HW testing: review 139927 (abandoned).
  • Flatten Aggregate Metadata in the DB: review 134573 (abandoned).
  • Flatten Instance Metadata in the DB: review 134945 (abandoned).
  • Implement a new code coverage API extension: review 130855.
  • Move flavor data out of the system_metadata table in the SQL database: review 126620 (approved).
  • Move to polling for cinder operations: review 135367.
  • PCI test cases for third party CI: review 141270.
  • Transition Nova to using the Glance v2 API: review 84887.
  • Transition to using glanceclient instead of our own home grown wrapper: review 133485 (approved).




Internationalization



  • Enable lazy translations of strings: review 126717 (fast tracked).




Networking







Performance



  • Dynamically alter the interval nova polls components at based on load and expected time for an operation to complete: review 122705.




Scheduler



  • A nested quota driver API: review 129420.
  • Add a filter to take into account hypervisor type and version when scheduling: review 137714.
  • Add an IOPS weigher: review 127123 (approved, implemented); review 132614.
  • Add instance count on the hypervisor as a weight: review 127871 (abandoned).
  • Allow extra spec to match all values in a list by adding the ALL-IN operator: review 138698 (fast tracked, approved).
  • Allow limiting the flavors that can be scheduled on certain host aggregates: review 122530 (abandoned).
  • Allow the remove of servers from server groups: review 136487.
  • Convert get_available_resources to use an object instead of dict: review 133728 (abandoned).
  • Convert the resource tracker to objects: review 128964 (fast tracked, approved).
  • Create an object model to represent a request to boot an instance: review 127610 (approved).
  • Decouple services and compute nodes in the SQL database: review 126895 (approved).
  • Enable adding new scheduler hints to already booted instances: review 134746.
  • Fix the race conditions when migration with server-group: review 135527 (abandoned).
  • Implement resource objects in the resource tracker: review 127609.
  • Improve the ComputeCapabilities filter: review 133534.
  • Isolate Scheduler DB for Filters: review 138444.
  • Isolate the scheduler's use of the Nova SQL database: review 89893.
  • Let schedulers reuse filter and weigher objects: review 134506 (abandoned).
  • Move select_destinations() to using a request object: review 127612 (approved).
  • Persist scheduler hints: review 88983.
  • Refactor allocate_for_instance: review 141129.
  • Stop direct lookup for host aggregates in the Nova database: review 132065 (abandoned).
  • Stop direct lookup for instance groups in the Nova database: review 131553 (abandoned).
  • Support scheduling based on more image properties: review 138937.
  • Trusted computing support: review 133106.




Scheduling







Security



  • Make key manager interface interoperable with Barbican: review 140144 (fast tracked, approved).
  • Provide a reference implementation for console proxies that uses TLS: review 126958 (fast tracked, approved).
  • Strongly validate the tenant and user for quota consuming requests with keystone: review 92507.




Service Groups







Sheduler



  • Add soft affinity support for server group: review 140017 (approved).




Tags for this post: openstack kilo blueprint spec nova

Related posts: Specs for Kilo; One week of Nova Kilo specifications; Compute Kilo specs are open; Specs for Kilo; Juno nova mid-cycle meetup summary: slots; Juno nova mid-cycle meetup summary: nova-network to Neutron migration



Comment

Michael Still: Soft deleting instances and the reclaim_instance_interval in Nova

Mon, 2014-12-15 09:29
I got asked the other day how the reclaim_instance_interval in Nova works, so I thought I'd write it up here in case its useful to other people.



First off, there is a periodic task run the nova-compute process (or the computer manager as a developer would know it), which runs every reclaim_instance_interval seconds. It looks for instances in the SOFT_DELETED state which don't have any tasks running at the moment for the hypervisor node that nova-compute is running on.



For each instance it finds, it checks if the instance has been soft deleted for at least reclaim_instance_interval seconds. This has the side effect from my reading of the code that an instance needs to be deleted for at least reclaim_instance_Interval seconds before it will be removed from disk, but that the instance might be up to approximately twice that age (if it was deleted just as the periodic task ran, it would skip the next run and therefore not be deleted for two intervals).



Once these conditions are met, the instance is deleted from disk.



Tags for this post: openstack nova instance delete

Related posts: One week of Nova Kilo specifications; Specs for Kilo; Juno nova mid-cycle meetup summary: nova-network to Neutron migration; Juno Nova PTL Candidacy; Juno nova mid-cycle meetup summary: scheduler; Juno nova mid-cycle meetup summary: ironic



Comment

Andrew McDonnell: Experiments with hardening OpenWRT: applying the grsecurity patches

Mon, 2014-12-15 00:28

A well known set of security enhancements to the Linux kernel is the grsecurity patch.  The grsecurity patch is a (large) patch that applies cleanly against selected supported stock Linux kernel versions. It brings with it PAX, which protects against various well known memory exploits, plus  a number of other hardening features including logging time and mount changes. In particular it enables features such as Non-executable stack (NX) on platforms that do not provide NX in hardware, such as MIPS devices and older x86.

OpenWRT hardening

OpenWRT is a widely adopted embedded / router Linux distribution. It would benefit greatly from including grsecurity, in particular given most MIPS platforms do not support NX protection in hardware. However for a long time the differences between the OpenWRT kernel and the kernel revisions that grsecurity is supported on have been significant and would likely have taken an extreme effort to get working, let alone get working securely.

This is a shame, because there is malware targeted at consumer embedded routers, and it must only be a matter of time before OpenWRT is targeted.  OpenWRT is widely regarded as relatively secure compared to many consumer devices, at least if configured properly,  but eventually some bug will allow a remote binary to be dropped. It would be helpful if the system can be hardened and stay one step ahead of things.

The OpenWRT development trunk (destined to become the next release, ‘Chaos Calmer’ in due course) has recently migrated most devices to the 3.14 kernel tree.  Serendipidously this aligns with the long term supported grsecurity revision 3.14.  When I noticed this I figured I’d take a look at whether it was feasible to deploy grsecurity with OpenWRT.

Applying grsecurity – patch

In late November I pulled the latest OpenWRT sources and the kernel version was 3.14.25, which I noticed matched the current grsecurity stable branch 3.14.25

The grsecurity patch applies cleanly against a stock kernel, and OpenWRT starts with a stock kernel and then applies a series of patches designed to extend hardware support to many obscure embedded things not present in the mainline kernel, along with patches that reduce the memory footprint. Some of the general patches are pushed upstream but may not yet have been accepted, and some could be backports from later kernels.  Examples of generic patches  include a simplified crash report.

Anyway, I had two choices, and tried them both: apply grsecurity, then the OpenWRT patches; or start with the OpenWRT patched kernel.  In both cases there were a number of rejects, but there seemed to be less when I applied grsecurity last. I also decided this would be easier for me to support for myself going forward, a decision later validated successfully.

OpenWRT kernel patches are stored in two locations; generic patches applying against any platform, then platform specific patches.  My work is tested against the Carambola2, an embedded MIPS board supported by the ‘ar71xx’ platform in OpenWRT, so for my case, there were ar71xx patches.

To make life easy I wrote a script that would take a directory of OpenWRT kernel patches, apply to a git kernel repository and auto-commit. This allowed me to use gitg and git difftool to examine things efficiently.  It also worked well with using an external kernel tree to OpenWRT so I didnt have to worry yet about integrating patches into OpenWRT. This script is on github, it should be easily adaptable for other experiments.

(Note: to use an external tree, managed by git, use config options like the following:

CONFIG_KERNEL_GIT_CLONE_URI="path/to/linux-stable" CONFIG_KERNEL_GIT_LOCAL_REPOSITORY="path/to/linux-stable" CONFIG_KERNEL_GIT_BRANCH="owrt_grsec_v3.14.25"

There were four primary rejects that required fixing.  This involved inspecting each case and working out what OpenWRT had changed in the way. Generally, this was caused because one or the other had modified the end of the same structure or macro, but luckily it turned out nothing significant and I was able to easily reconcile things. The hardest was because OpenWRT modifies vmstat.c for MIPS and the same code was modified by grsecurity to add extra memory protections.  At this point I attempted to build the system, and discovered three other minor cases that broke the build. These mispatches essentially were due to movements in one or two lines, or new code using internal kernel API modified by grsecurity, and were also easily repaired.  The most difficult mispatch to understand was where OpenWRT rewrites the kernel module loader code, apparently to make better use of MIPS memory structures and it took me a little while to understand how to try and fix things.

The end result is on github at https://github.com/pastcompute/openwrt-cc-linux-3.14.x-grsecurity

Applying grsecurity – OpenWRT quirks

One strange bug that had to be worked around was some new dependency in the kernel build process, where extra tools that grsecurity adds were not being built in the correct order with other kernel prerequisites.

In the end I had to patch how OpenWRT builds the kernel to perform an extra ‘make olddefconfig‘ to sort things out.

I also had to run ‘make kernel_menuconfig‘ and turn on grsecurity.

As the system built, I eventually hit another problem area: building packages. This was a bit of an ‘OH-NO’ moment as I thought it had the potential to become a big rabbit hole. Luckily as it turned out, only one package was affected in the end: compat-wireless.  This package builds some extra user space tools and wifi drivers, and used a macro, ACCESS_ONCE, that was changed by grsecurity to be more secure; and required use of a new macro to make everything work again, ACCESS_ONE_RW. There were rather a number of calls to this macro, but luckily it turned out to be fixable using sed!

Booting OpenWRT with grsecurity – modules not loading

I was able to then complete an INITRAMFS image that I TFTP’d into my carambola2 via uboot.

Amazingly the system booted and provided me with a prompt.

U-Boot 1.1.4-g33f82657-dirty (Sep 16 2013 - 16:09:28) ===================================== CARAMBOLA2 v1.0 (AR9331) U-boot   Starting kernel ... [ 0.000000] Linux version 3.14.26-grsec (andrew@atlantis4) (gcc version 4.8.3 (OpenWrt/Linaro GCC 4.8-2014.04 r43591) ) #3 Sun Dec 14 18:08:52 ACDT 2014

I then discovered that no kernel modules were loading. A bit of digging and it turns out that a grsecurity option, CONFIG_GRKERNSEC_RANDSTRUCT  will auto-enable CONFIG_MODVERSIONS. One thing I learned at this point is that OpenWRT does not support CONFIG_MODVERSIONS=y, due to the way it packages modules with its packaging system. So an iteration later with the setting disabled, and everything appeared to be “working”

Testing OpenWRT with grsecurity

Of course, all this work is moot if we cant prove it works.

Easy to check is auditing. For example, we now had these messages:

[ 4.020833] grsec: mount of proc to /proc by /sbin/init[init:1] uid/euid:0/0 gid/egid:0/0, parent /[swapper:0] uid/euid:0/0 gid/egid:0/0 [ 4.020833] grsec: mount of sysfs to /sys by /sbin/init[init:1] uid/euid:0/0 gid/egid:0/0, parent /[swapper:0] uid/euid:0/0 gid/egid:0/0 [ 4.041666] grsec: mount of tmpfs to /dev by /sbin/init[init:1] uid/euid:0/0 gid/egid:0/0, parent /[swapper:0] uid/euid:0/0 gid/egid:0/0

However, the acid test would be enforcement of the NX flag. Here I used the code from http://wiki.gentoo.org/wiki/Hardened/PaX_Quickstart to test incorrect memory protections. Result:

[19111.666360] grsec: denied RWX mmap of <anonymous mapping> by /tmp/bad[bad:1497] uid/euid:0/0 gid/egid:0/0, parent /bin/busybox[ash:467] uid/euid:0/0 gid/egid:0/0 mmap failed: Operation not permitted

Success!

Revisiting Checksec, and tweaking PAX

In an earlier blog I wrote about experimenting with checksec.  Here I used it to double-check that the binaries were built with NX protection. MOst were, due to a patch I previously submitted to OpenWRT for MIPS. However, openssl was missing NX. It turns out that OpenSSL amongst everything else it has been discussed for this year, uses assembler in parts of the encryption code! I was able to fix this by adding the relevant linker ‘.note.GNU-stack‘ directive.

The PAX component can be tweaked using the paxctl command, so I had to build that with the OpenWRT toolchain to try it out. I discovered that it doesnt work for files on the JFFS2 partition, only in the ramdisk. Further to enable soft mode, you need to add a kernel boot command line argument. To do this for OpenWRT, edit a file called target/linux/$KERNEL_PLATFORM/generic/config-default where in my case, $KERNEL_PLATFORM is ar71xx

Moving Targets

Right in the middle of all this, OpenWRT bumped the kernel to 3.14.26. So I had to exercise a workflow in keeping the patch current.  As it happened the grsecuroty patch was also updated to 3.14.26 so I presume this made life easier.

After downloading the stock kernel and pulling the latest OpenWRT, I again re-created the patch series, then applied grsecurity 3.14.26.  The same four rejects were present again, so fingers crossed I cherry-picked all my work from 3.14.25 onto 3.14.26. As luck would have it this was one smooth rebase!

Recap of OpenWRT grsecurity caveats
  • CONFIG_GRKERNSEC_RANDSTRUCT is not compatible with the OpenWRT build system; using it will prevent modules loading
  • Some packages may need to be modified to support NX – generally, if these use assembly language and don’t use the proper linker directive.
  • For some reason paxctl only seems to work on files in /tmp not in the JFFS overlay. This is probably only a problem when debugging
  • Your experience with the debugger gdb will probably be sub-optimal unless you put the debug target on /tmp and use paxctl to mark it with exceptions
Summary

After concluding the above, I converted the change set from my local Linux working copy into a set of additional patches on OpenWRT and rebuilt everything to double check.

The branch ‘ar71xx-3.14.26-grsecurity’ in https://github.com/pastcompute/openwrt-cc-ar71xx-hardened has all the work, along with some extra minor fixes I made to some other packages related to checksec scan results.

THIS MAY EXPLODE YOUR COMPUTER AND GET YOU POWNED! This has been working for me on one device with minimal testing and is just a proof of concept.

David Rowe: FSK over FM

Sun, 2014-12-14 10:30

I’m interested in developing a VHF mode for FreeDV. One intriguing possibility is to connect a modem to legacy analog FM radios, which would allow them to be re-purposed for digital voice. One candidate is FSK at 1200 bit/s, which is often used over FM for APRS. This operates through FM radios using the mic/speaker ports on $50 HTs, no special data ports required.

So I want to know the performance of FSK over FM in terms of Bit Error Rate (BER) for a given SNR. That got me thinking. When you send FSK through a SSB radio, it faithfully mixes the tones up to RF and you get FSK over the channel. The SSB radio just adds a frequency translation step. So we can model FSK like this:

However sending a FSK modem signal through a FM radio is very different:

FSK over FM is not FSK when you look at the over the air waveform. The spectrum is no longer two tones bouncing back and forth. So what is it?

I wrote a simulation called fsk.m to find out. This involved building up a FSK modem, and an analog FM radio simulation. The modem took me only a few hours but I was struggling with the analog FM simulation for a week! In particular making my FM demodulator get the same results as the theory. FM is a bit old school for me, so I had to hit the ARRL handbook and do a bit of research.

FSK Modem

It’s a BEL202 simulation (as used for the APRS physical layer); 1200/2200 Hz tones, 1200 bit/s. I’m using the integrate and dump demodulation method and it matches the theoretical curves for non-coherent BFSK. Here is the FSK modem in action. First the FSK time domain signal and spectrum. The spectrum is a bunch of energy between 1200 and 2200 Hz. Makes sense as the modulator keeps moving back and forth between those two frequencies.

The next figure shows the sames signals with a 10dB SNR. Although the time domain signal looks bad, it actually has a BER one error in every 1000 bits (1E-3). The reason it looks so bad is that in the time domain we are seeing the noise from the entire bandwidth (our sample rate is Fs=96kHz). The demod effectively filters most of that out.

This next plot shows the output from the 1200 and 2200Hz integrators in the FSK demodulator for the 10dB SNR case. The height measures the energy of the tone during that bit period. As we would expect, they are mirror images. When one detects a large amount of energy, the other detects a small amount of the other tone.

Analog FM

The next step was to build a simulation of the modulator and demodulator in an analog FM radio. I wrote some code to test the input Carrier to Noise Ratio (CNR) versus output SNR. The test signal was a 1000 Hz tone, and the modulator had a maximum deviation of 5kHz, and a maximum input audio frequency of 3 kHz. After the demodulator I notched out the 1000 Hz tone so I could measure the noise power, the input to the notch filter was signal plus noise.

Here is the spectrum at the FM demodulator input for a 1000 Hz test tone:

The top plot is the tx signal centred on a 24 kHz carrier, in the bottom plot it has been mixed down to baseband and filtered. The FM signal is 16 kHz wide, as per Carsons rule. Here is the output of the FM demodulator:

At the top is a nice sine wave, and the bottom also shows the sine wave. You can see the effect of the output 3kHz low pass filter used to limit the noise bandwidth of the demod output.

When tested over a range of CNR inputs, I achieved output SNRs (red) in line with the text books (green):

At about 9dB the demodulator falls away from theory as the FM demodulator falls over, this is pretty typical. The theoretical model I have used is only valid above this 9dB threshold. You often hear this threshold effect in FM. The blue line is SSB for comparison. Over a certain threshold FM does quite a bit better in terms of output SNR for the same input CNR.

FSK over FM

OK so lets combine the simulations and look at the BER performance:

Oh dear. If my simulations are accurate, it appears FSK over FM is a lemon. About 7dB worse than regular FSK for the same BER. So using a FSK modem over a SSB radio would allow you to use 7dB less power than running the same modem through a FM radio. Coherent PSK is 3dB better again that FSK so that would get you a 10dB improvement. Simple FSK or PSK transmitters are easy to build too, and needing 7-10dB less output power would simplify them again (e.g. 100mW versus 1W).

Here is the spectrum at the FM demodulator input when sending FSK:

Note the FM spectrum looks nothing like regular FSK “over the air”, which looks like this:

So What went Wrong?

Given the plot of analog FM performance (say compared to SSB) above I had expected better results from FSK over FM.

I think I know where the problem lies. The input CNR is a measure of carrier power to noise power in the input bandwidth of the demodulator. Another way of looking at the VHF channel noise is a “floor”, which can be modelled as the average noise power per 1 Hz of bandwidth, called No.

So the Universe has given us a fixed “noise floor”, which will be the same for any modem. The FM demod input bandwidth is much wider, so it’s sucking up much more noise from the channel, which the poor demodulator has to deal with.

Lets plot the analog FM demod performance again, this time against C/No rather than C/N:

This takes into account the noise bandwidth, everything is “normalised” to the noise floor. When the C/No is beneath 48dB SSB looks much better. We can see a 7dB improvement over FM at low C/No values. This also explains why the microwave guys prefer SSB for their long shots.

Here is the BER curve scaled for C/No:

Conclusion

It appears the key to good modem performance is the RF bandwidth of the signal. Given a constant noise floor No, the signal bandwidth sets the total noise power N=NoB the demodulator has to deal with.

This has put me off the idea of a FreeDV VHF mode based on BEL202 FSK through legacy FM radios. I’d really like to come up with a mode that has sparkling BER versus SNR performance. I haven’t spent years making Codec 2 operate at low bit rates just to throw those gains away in the modem!

Couple of ways forward:

  • Take a look at GMSK.
  • Consider developing a version of the SM1000 into an (open source) VHF SDR radio that can do PSK. Not as crazy as it seems. We are already planning a HF SDR version. Radio hardware is getting simple now the signal processing is all moving to software. We can make the modem so efficient that the PA can be modest (100s of mW).
  • Dream up waveforms that can pass through legacy FM radios and have a low over-the air bandwidth. For example FSK that shifts between 300 and 400 Hz. In the past I’ve dreamed up new Codec 2 modes (1300 and 450 bit/s) to suit the properties of HF channels. So why not design a modem waveform to suit us? Go open source!
  • Cop the performance hit and use BEL202 FSK. It might still be useful to use legacy FM radios for DV even with a 7dB loss in modem performance. It seems to work fine for APRS. If your C/No is high (as is often the case) then FSK over FM will have zero errors.

Andrew Pollock: [life] Day 317: Doctor again, final Tumble Tastics, a good deed and general fun

Sat, 2014-12-13 12:26

Zoe slept solidly until 6:48am. It was overcast and cooler, so I dare say that helped. Uninterrupted sleep is always nice. We had a nice snuggle in bed before we started the day.

First up, we had another doctor's appointment so the doctor could have another go at freezing off the wart on her hand. Despite some initial uncertainty, Zoe was much braver this time, and the doctor got to really hit it this time. Zoe was very proud of herself.

After the obligatory Freddo Frog for bravery, we headed home via the Valley to clear my PO box.

After a little bit of TV, we scootered to Tumble Tastics for her final class.

Tumble Tastics has been really great for Zoe. Zoe's always enjoyed gymnastics, and has definitely enjoyed this. She was very fond of Mr Fletcher, her teacher (she seems to really like male teachers) and especially loved the rope swing they had in the classroom. I was personally impressed by the theme that they did each week, and their ability to keep the activities in the relatively small room fresh and varied each week. They use the limited space that they have quite effectively. The fact that it was an easy distance from home was a bonus.

On our way back home, we discovered a stray dog on the side of Hawthorne Road. I checked its collar, and it had a mobile phone number on it, so I gave it a call. It turned out the owner was down at the supermarket, and his wife was at home with a baby, so I offered to return the dog for him.

It was only about a 500 metre walk, but it was very back-breaking, as the dog was pretty dumb and wouldn't follow us, so I head to lead it by the collar all the way, which involved me having to walk bent over all the way. Zoe wanted to help, but he was a bit to big and heavy for her to lead.

He was an interesting cross-breed. He had the markings of a blue heeler, but the head and general body shape of a terrier of some sort.

Due to some ambiguous letterboxes, we ended up at the the wrong house (off by one) and this house had a black Siamese cat that emerged from a boat parked in the front yard when I knocked on the door. Of course the dog decided to chase off after the cat, and I thought all was lost at that point, but he came back after having chased the cat away.

We then proceeded to the right house, returned the dog and went home for a well earned lunch.

After lunch, we went for a walk in the rain to post a letter. Zoe had a great time puddle jumping in her rain boots. We also made an opportunistic Christmas present purchase, and then went home again.

We had an unplanned afternoon of silly play for a while, with lots of running around and tickles and laughter. It was nice. Our downstairs neighbour, Deana, popped up to hang out for a bit as well, which was nice.

Zoe watched a bit of TV after that, and then Sarah arrived to pick her up.

Glen Turner: USB product IDs for documentation - success

Fri, 2014-12-12 23:47
Motivation

In a previous posting I reported a lack of success when enquiring of the USB Implementors' Forum if a Vendor ID had been reserved for documentation.

To recap my motivation, a Vendor ID -- or at least a range of Product IDs -- is desirable to:

  • Avoid defamation, such as using a real VID:PID to illustrate a "workaround", which carries the implication that the product is less-than-perfect. Furthermore, failing to check if a VID:PID has actually been used is "reckless defamation".

  • Avoid consumer law, such as using a real VID:PID to illustrate a a configuration for a video camera, when in fact the product is a mouse.

  • Avoid improper operation, as may occur if a user cuts-and-pastes an illustrative example and that effects a real device.

  • Avoid trademark infringment.

For these reasons other registries of numbers often reserve entries for documentation: DNS names, IPv4 addresses, IPv6 addresses.

Allocation of 256 Product IDs, thanks to OpenMoko

OpenMoko has been generous enough to reserve a range of Product IDs for use by documentation:

0x1d50:0x5200 through to 0x1d50:0x52ff

Note carefully that other Product IDs within Vendor ID 0x1d50 are allocated to actual physical USB devices. Only the Product IDs 0x1d50:0x5200 through to 0x1d50:0x52ff are reserved for use by documentation.

My deep thanks to OpenMoko and Harald Welte.

Application form

The application form submitted to OpenMoko read:

  • a name and short description of your usb device project

    Documentation concerning the configuration of USB buses and devices.

    For example, documentation showing configuration techniques for Linux's udev rules.

    The meaning of "documentation" shall not extend to actual configuration of a actual device. It is constrained to showing methods for configuration. If an VID:PID for an actual device is required then these can be obtained from elsewhere.

    OpenMoko will not assign these "Documentation PIDs" to any actual device, now or forever.

    Operating systems may refuse to accept devices with these "documentation VID:PIDs". Operating systems may refuse to accept configuration which uses these "documentation VID:PIDs".

  • the license under which you are releasing the hardware and/or software/firmware of the device

    The documentation may use any license. Restricting use to only free documentation is problematic: the definition of "free" for documents is controversial; and it would be better if the PID:VIDs were well known and widely used by all authors of technical documentation.

  • a link to the project website and/or source code repository, if any

    Nil, one can be created if this is felt to be necessary (eg, to publicise the allocation).

  • if you need multiple Product IDs, please indicate + explain this at the first message, rather than applying for a second ID later

    Approximately 10.

Andrew Pollock: [life] Day 316: Bike riding play date and picnic

Thu, 2014-12-11 22:25

Zoe woke up at around 1:30am. I think the fact that her nightlight had gotten unplugged didn't help matters, and despite fixing that up, she jumped into bed with me at 1:50am.

We had a slow start to the day, but that said, I did manage to bake a batch of mince pies and make pastry for a quiche before we headed out at 9:30am, so it wasn't an unproductive morning.

I'd organised with Kelley to have a bike riding play date with Chloe at the Minnippi Parklands. I figured that since Chloe can already ride a bike, it might encourage Zoe.

It was a pretty hot morning, and not a lot of attempted bike riding happened before Zoe had had enough. No major breakthroughs happened, but it was very handy having a second adult. I think I need to put Zoe's bike seat up, as she's grown a bit since she first started trying to learn.

After we gave up on the bikes, the girls went and played on the pretend aeroplane and air traffic control tower for the rest of the morning, and we watched a storm roll in.

By early afternoon, the storm was looking a bit ominous, and Kelley had to be back at school, so we dropped them back home, and Zoe played for a bit at Chloe's place before we headed home to get ready for swim class.

In the mean time, the storm hit and appeared to pass, so we drove to swim class, but there was still lightning around, so swim class was canceled.

We headed back home so I could finish making dinner. Zoe was pretty tired from the day's activities, so I'm hoping she has a good sleep tonight.

Andrew Pollock: [life] Day 315: End of Kindergarten

Thu, 2014-12-11 21:25

Well, the day finally arrived. Zoe graduated from Kindergarten.

I started the morning failing to go for a run again. I should just give up until after Summer I think. I had my chiropractic adjustment and did some blogging, before heading out to Zoe's Kindergarten for their farewell morning tea.

The morning tea was nice, and we were free to take our kids home afterwards, so we were out of there by about midday.

The Kindergarten year feels like it's flown by so quickly. I've been really happy with the Seven Hills C&K. I'm grateful my friend Kim told me about C&K while there was still time to get Zoe a place, and that I've been able to have the year off to allow her to go. I think it's definitely a better environment than long day care. The transition statement, which I guess can best be described as an exit report card for Zoe was absolutely lovely. Time will tell, but I think it has prepared her well for Prep next year.

It was also great to be able to serve on the Kindergarten's Parent Advisory Group committee this year. That gave some insight into how the Kindergarten, and C&K in general operate.

Seven weeks of school holidays lie ahead of us, which I'm sure will fly by pretty quickly.

Megan came over with her Mum in the afternoon for a play date, and I prepared some more mince pies.

Zoe and I went out for dinner with my cousin, Renata, to celebrate Zoe's graduation from Kindergarten and we had a nice dinner out.

David Rowe: SM1000 Part 9 – First Betas

Thu, 2014-12-11 18:29

Edwin and his team at Dragino have hand assembled the first two SM1000 Beta units in Shenzhen. I’m working with him to perform some initial tests while we wait a few days for the prototype enclosures to be made. Then Rick and I will both get a SM1000 shipped to us for testing. We’ve found a few little issues so far which we will correct before the Qty 100 beta run later this month.







Stewart Smith: skiboot-4.1

Thu, 2014-12-11 17:26

I just posted this to the mailing list, but I’ve tagged skiboot-4.1, so we have another release! There’s a good amount of changes since 4.0 nearly a month ago and this is the second release since we hit github back in July.

For the full set of changes, “git log” is your friend, but a summary of them follows:

  • We now build with -fstack-protector and -Werror
  • Stack checking extensions when built with STACK_CHECK=1
  • Reduced stack usage in some areas, -Wstack-usage=1024 now.
    • Some functions could use 2kb stack, now all are <1kb
  • Unsafe libc functions such as sprintf() have been removed
  • Symbolic backtraces
  • expose skiboot symbol map to OS (via device-tree)
  • removed machine check interrupt patching in OPAL
  • occ/hbrt: Call stopOCC() for implementing reset OCC command from FSP
  • occ: Fix the low level ACK message sent to FSP on receiving {RESET/LOAD}_OCC
  • hardening to errors of various FSP code
    • fsp: Avoid NULL dereference in case of invalid class_resp bits
    • abort if device tree parsing fails
    • FSP: Validate fsp_msg in fsp_queue_msg
    • fsp-elog: Add various NULL checks
  • Finessing of when to use error log vs prerror()
  • More i2c work
  • Can now run under Mambo simulator (see external/mambo/skiboot.tcl) (commonly known as “POWER8 Functional Simulator”)
  • Document skiboot versioning scheme
  • opal: Handle more TFAC errors.
    • TB_RESIDUE_ERR, FW_CONTROL_ERR and CHIP_TOD_PARITY_ERR
  • ipmi: populate FRU data
  • rtc: Add a generic rtc cache
  • ipmi/rtc: use generic cache
  • Error Logging backend for bmc based machines
  • PSI: Drive link down on HIR
  • occ: Fix clearing of OCC interrupt on remote fix

So, who worked on this release? We had 84 csets from 17 developers. A total of 3271 lines were added, 1314 removed (delta 1957).

Developers with the most changesets Stewart Smith 24 28.6% Benjamin Herrenschmidt 17 20.2% Alistair Popple 8 9.5% Vasant Hegde 6 7.1% Ananth N Mavinakayanahalli 5 6.0% Neelesh Gupta 4 4.8% Mahesh Salgaonkar 4 4.8% Cédric Le Goater 3 3.6% Wei Yang 3 3.6% Anshuman Khandual 2 2.4% Shilpasri G Bhat 2 2.4% Ryan Grimm 1 1.2% Anton Blanchard 1 1.2% Shreyas B. Prabhu 1 1.2% Joel Stanley 1 1.2% Vaidyanathan Srinivasan 1 1.2% Dan Streetman 1 1.2% Developers with the most changed lines Benjamin Herrenschmidt 1290 35.1% Alistair Popple 963 26.2% Stewart Smith 344 9.4% Mahesh Salgaonkar 308 8.4% Ananth N Mavinakayanahalli 198 5.4% Neelesh Gupta 186 5.1% Vasant Hegde 122 3.3% Shilpasri G Bhat 39 1.1% Vaidyanathan Srinivasan 24 0.7% Joel Stanley 21 0.6% Wei Yang 20 0.5% Anshuman Khandual 15 0.4% Cédric Le Goater 12 0.3% Shreyas B. Prabhu 9 0.2% Ryan Grimm 3 0.1% Anton Blanchard 2 0.1% Dan Streetman 2 0.1% Developers with the most lines removed Mahesh Salgaonkar 287 21.8% Developers with the most signoffs (total 54) Stewart Smith 44 81.5% Vasant Hegde 4 7.4% Benjamin Herrenschmidt 4 7.4% Vaidyanathan Srinivasan 2 3.7% Developers with the most reviews (total 2) Vasant Hegde 2 100.0%

linux.conf.au News: Keynote Speaker - Bob Young

Thu, 2014-12-11 07:28

The LCA 2015 team are honoured to announce our third Keynote speaker - Bob Young, founder and chairman of Lulu.com, co-founder of Red Hat and the Center for Public Domain.

Bob's presentation is scheduled for 09:00 am Wednesday, 14 January 2015

Bob Young is the founder and chairman of Lulu.com, a premiere international marketplace for new digital content on the Internet, with more than 300,000 recently published titles and more than 15,000 new creators from 80 different countries joining each week.

Lulu.com, founded in 2002, is Young's most recent endeavour. The success of this company has earned Young notable recognition; he was named one of the "Top 50 Agenda-Setters in the Technology Industry in 2006" and was ranked as the fourth "Top Entrepreneur for 2006," both by Silicon.com.

In 1993, Young co-founded Red Hat (NYSE: RHT), the open-source software company that gives hardware and software vendors a standard platform on which to certify their technology. Red Hat is a Fortune 500 company and chief rival to Microsoft. His success at Red Hat won him industry accolades, including nomination as one of Business Week's "Top Entrepreneurs" in 1999

Before founding Red Hat, Young spent 20 years at the helm of two computer-leasing companies that he founded. His experiences as a high tech entrepreneur combined with his innate marketing savvy led to Red Hat's success. His book, "Under the Radar", chronicles how Red Hat's open source strategy successfully won wide industry acceptance in a market previously dominated by proprietary binary-only systems. Young has also imparted the lessons learned from his entrepreneurial experiences through his contributions to the books to "You've GOT to Read This Book!" and "Chicken Soup for the Entrepreneur's Soul."

In 2000, Young co-founded the Center for Public Domain, a non-profit foundation created to bolster healthy conversation of intellectual property, patent and copyright law, and the management of the public domain for the common good. Grant recipients included the Electronic Frontier Foundation, the Creative Commons, the Free Software Foundation, and the Future of Music Coalition.

In addition to enjoying fly fishing, Young collects calculators and antique typewriters, a nod to his beginnings as a typewriter salesman and can usually be found sporting a pair of red socks. However, instead of red on his head, Young now tips his orange hat.

The LCA 2015 Auckland Team

Andrew Pollock: [life] Day 314: Kindergarten, startup stuff

Wed, 2014-12-10 11:26

I wanted to start the day with a run, but like so many mornings before this one, I just couldn't be bothered. I've really fallen out of the habit.

Instead, I managed to knock over another unit of my real estate licence course by 9:30am. It turned out to be a short and sweet unit, which was great. I'm happy to be down to only four units to go by the end of Kindergarten. It's not the highly optimistic progress that I'd hoped, but being down to the last quarter of the course is still a nice milestone to go into the school holidays at.

To atone for my lack of running, I ended up wandering all over the neighbourhood instead. First in one direction to drop a dish back to Kelley, then in another direction to the post office to mail off the assessment, and then back the other way again to put a brochure in a prospect's mailbox. Google Fit was very happy with me.

After doing a bunch of random stuff at home, I caught a bus into the city for a lunch meeting, and then a taxi home in time to pick Zoe up from Kindergarten.

We popped out to Bunnings to pick up a tap timer for the common area gardens, and then got home before the storm of the day hit.

I did a spot of baking for Zoe's Kindergarten end of year morning tea, and Sarah arrived to pick Zoe up.

The storm didn't really produce much other than a spectacular light show. I haven't seen so much lightning since being back in Brisbane, and it was very impressive. I did miss a good electrical storm living in California...

I closed out the day with a nice yoga class.

Linux Users of Victoria (LUV) Announce: LUV Beginners December Meeting: SUSE Linux

Tue, 2014-12-09 20:29
Start: Dec 20 2014 12:30 End: Dec 20 2014 16:30 Start: Dec 20 2014 12:30 End: Dec 20 2014 16:30 Location: 

RMIT Building 91, 110 Victoria Street, Carlton South

Link:  http://luv.asn.au/meetings/map

Terry Kemp will demonstrate installation and provide an overview of Open SUSE 13.2 and if time allows an overview of SUSE Studio.

Terry is a member of LUV and long-time Linux user from the early days of Slackware -- until one day he saw at a LUV install fest at MLC an install of SUSE, which he has been using since. He was on the committee of the Melbourne PC Users Group and convenor of their Linux and Open Souce SIG.

LUV would like to acknowledge Red Hat for their help in obtaining the Buzzard Lecture Theatre venue and VPAC for hosting.

Linux Users of Victoria Inc., is an incorporated association, registration number A0040056C.

December 20, 2014 - 12:30

read more

Andrew Pollock: [life] Day 313: Kindergarten, errands and podiatrist

Tue, 2014-12-09 12:25

I had a pretty busy morning. Zoe woke up around 12:45am complaining about a sore foot or something. I never quite got to the bottom of it, but I resettled her and got her back to sleep.

She then woke up at 6am for the day, and we managed to breeze through the morning super easily (for a change), with enough time for me to put laundry away and pump up the bike tyres and still arrive at Kindergarten right on opening time. It was all rather amazing.

The Kindergarten director asked me if I happened to have 40 CD sleeves "because I was a computer person", so I offered to go pick some up for her.

I biked back home, and fixed up one of my real estate licence course units that I had to resubmit due to a couple of mistakes, and then headed out in the car.

I debated going to OfficeWorks, which would have resulted in more back and forth, or trying K Mart at Cannon Hill. I ended up going for K Mart, and they didn't have any. Neither did Dick Smith or the post office. I got lucky at one of the cheap shops, and managed to get a pack of 100. I did manage to pick up some cheap plain t-shirts for Zoe to wear under her sun dresses to protect her shoulders, so it wasn't a completely wasted trip.

I dropped the sleeves back into the Kindergarten, and then mailed off the corrected coursework and headed back home.

There was another storm looming at around pick up time, so I drove back to Kindergarten. They've already started end of year clean up stuff, and they've taken down all the stuff from this year from the walls, and the Kindergarten looks shocking stark now.

My health insurance covers podiatry, so I figured I might as well get Zoe checked out before we buy school shoes. I'd made an appointment for 3pm at the myFootDr headquarters over in Camp Hill, where my neighbour Meg happens to work.

Zoe was initially not keen on going, but once we got there, she was fine. They were running a bit behind, so we watched some TV in the waiting room. After watching Wild Vets, Zoe's decided she doesn't want to be a vet any more.

We had a really nice podiatrist, and Zoe was very comfortable with her, and happy to go through the examination. The podiatrist was very impressed with Zoe's physical abilities and general balance, and said she was well ahead of her milestones, which was pretty impressive. I chalk that up to all of the varied physical activities she's done this year.

She said normally she wouldn't try and put a 4 and a half year old on the treadmill for video gait analysis, but given how good Zoe had been, she gave it a shot. Zoe did fine.

I'm really glad that I took her to the podiatrist, because I got some good general advice about what to expect with Zoe's growth given her body shape, and we bought a more appropriate shoe type for her foot than I'd have otherwise bought at the uniform shop. I also got a pair of running shoes for her as well. Zoe had heaps of fun to boot.

The storm hit while we were there, but fortunately there was no hail in it. It had mostly receded by the time we left. We popped in to the post office on the way home, and then the weather turned a bit foul again, and we timed it just right to pick up Anshu from the ferry terminal on the way home.

Sarah arrived not long after that to pick Zoe up.

Andrew Pollock: [tech] A geek Dad goes to Kindergarten with a box full of Open Source and some vegetables

Tue, 2014-12-09 12:25

Zoe's Kindergarten encourages parents to come in and spend some time with the kids. I've heard reports of other parents coming in and doing baking with the kids or other activities at various times throughout the year.

Zoe and I had both wanted me to come in for something, but it had taken me until the last few weeks of the year to get my act together and do something.

I'd thought about coming in and doing some baking, but that seemed rather done to death already, and it's not like baking is really my thing, so I thought I'd do something technological. I just wracked my brains for something low effort and Kindergarten-age friendly.

The Kindergarten has a couple of eduss touch screens. They're just some sort of large-screen with a bunch of inputs and outputs on them. I think the Kindergarten mostly uses them for showing DVDs and hooking up a laptop and possibly doing something interactive on them.

As they had HDMI input, and my Raspberry Pi had HDMI output, it seemed like a no-brainer to do something using the Raspberry Pi. I also thought hooking up the MaKey MaKey to it would make for a more fun experience. I just needed to actually have it all do something, and that's where I hit a bit of a creative brick wall.

I thought I'd just hack something together where based on different inputs on the MaKey MaKey, a picture would get displayed and a sound played. Nothing fancy at all. I really struggled to get a picture displayed full screen in a time efficient manner. My Pi was running Raspbian, so it was relatively simple to configure LightDM to auto-login and auto-start something. I used triggerhappy to invoke a shell script, which took care of playing a sound and an image.

Playing a sound was easy. Displaying an image less so, especially if I wanted the image loaded fast. I really wanted to avoid having to execute an image viewer every time an input fired, because that would be just way too slow. I thought I'd found a suitable application in Geeqie, because it supported being out of band managed, but it's problem was it also responded to the inputs from the MaKey MaKey, so it became impossible to predictably display the right image with the right input.

So the night before I was supposed to go to Kindergarten, I was up beating my head against it, and decided to scrap it and go back to the drawing board. I was looking around for a Kindergarten-friendly game that used just the arrow keys, and I remembered the trusty old Frozen Bubble.

This ended up being absolutely perfect. It had enough flags to control automatic startup, so I could kick it straight into a dumbed-down full screen 1 player game (--fullscreen --solo --no-time-limit)

The kids absolutely loved it. They were cycled through in groups of four and all took turns having a little play. I brought a couple of heads of broccoli, a zucchini and a potato with me. I started out using the two broccoli as left and right and the zucchini to fire, but as it turns out, not all the kids were as good with the "left" and "right" as Zoe, so I swapped one of the broccoli for a potato and that made things a bit less ambiguous.

The responses from the kids were varied. Quite a few clearly had their minds blown and wanted to know how the broccoli was controlling something on the screen. Not all of them got the hang of the game play, but a lot did. Some picked it up after having a play and then watching other kids play and then came back for a more successful second attempt. Some weren't even sure what a zucchini was.

Overall, it was a very successful activity, and I'm glad I switched to Frozen Bubble, because what I'd originally had wouldn't have held up to the way the kids were using it. There was a lot of long holding/touching of the vegetables, which would have fired hundreds of repeat events, and just totally overwhelmed triggerhappy. Quite a few kids wanted to pick up and hold the vegetables instead of just touch them to send an event. As it was, the Pi struggled to play Frozen Bubble enough as it was.

The other lesson I learned pretty quickly was that an aluminium BBQ tray worked a lot better as the grounding point for the MaKey MaKey than having to tether an anti-static strap around each kid's ankle as they sat down in front of the screen. Once I switched to the tray, I could rotate kids through the activity much faster.

I just wish I was a bit more creative, or there were more Kindergarten-friendly arrow-key driven Linux applications out there, but I was happy with what I managed to hack together with a fairly minimal amount of effort.

Andrew Pollock: [life] Day 310: Doctor, shopping, Tumble Tastics and a washed out Movies in the Park

Sun, 2014-12-07 22:25

Zoe slept solidly until 7am, which was lovely.

I'd made an appointment with Zoe's doctor to have some funny looking spots on her chest looked at. The spots were nothing, but the doctor decided to try and freeze the wart off Zoe's hand, which I hadn't been prepared for. Zoe was very brave about it though. The doctor wants to give it another hit in a week.

I've been wanting to get a portable shade tent for the beach for a while, and at BCF it's a seasonal item, so I had to wait for them to come into season. We headed over to BCF after the doctor.

BCF really is a poor cousin to REI. I miss REI. We did run into one of Zoe's Kindergarten teachers at BCF, so that was a bit of a treat for Zoe.

By the time we were done with that, it was getting close to time for Tumble Tastics, so we headed back home to get ready, and then Zoe wanted to go by car because it was hot.

We just chilled out at home after lunch, and I was watching the weather radar while debating the virtues of trying to go to the Hawthorne Markets and Councillor Shayne Sutton's last Movies in the Park night for the year. There was a pretty decent looking storm brewing.

In the end, I decided to wing it, and we headed off to the markets with umbrellas and blankets and camp chairs. I felt like a bit of a pack horse. As luck would have it, it started to rain a little bit half way there, and more earnestly once we arrived.

The upside was the line for free facepainting was non-existent, and Zoe also got to do some free craft activities without too much competition. Santa arrived not long after that.

The movie screening was moved inside the Morningside Panthers clubhouse, but the weather had turned most people away. There were probably only about 40 die hards that bothered to stick around.

The movie was Arthur Christmas, which wasn't something I was familiar with. It was pretty good though, if a little over Zoe's head. She enjoyed it nevertheless.

It was still raining lightly at the end of the movie, so we walked home with our umbrellas, and I got Zoe to bed.

Russell Coker: BTRFS Status Dec 2014

Fri, 2014-12-05 19:27

My last problem with BTRFS was in August [1]. BTRFS has been running mostly uneventfully for me for the last 4 months, that’s a good improvement but the fact that 4 months of no problems is noteworthy for something as important as a filesystem is a cause for ongoing concern.

A RAID-1 Array

A week ago I had a minor problem with my home file server, one of the 3TB disks in the BTRFS RAID-1 started giving read errors. That’s not a big deal, I bought a new disk and did a “btrfs replace” operation which was quick and easy. The first annoyance was that the output of “btrfs device stats” reported an error count for the new device, it seems that “btrfs device replace” copies everything from the old disk including the error count. The solution is to use “btrfs device stats -z” to reset the count after replacing a device.

I replaced the 3TB disk with a 4TB disk (with current prices it doesn’t make sense to buy a new 3TB disk). As I was running low on disk space I added a 1TB disk to give it 4TB of RAID-1 capacity, one of the nice features of BTRFS is that a RAID-1 filesystem can support any combination of disks and use them to store 2 copies of every block of data. I started running a btrfs balance to get BTRFS to try and use all the space before learning from the mailing list that I should have done “btrfs filesystem resize” to make it use all the space. So my balance operation had configured the filesystem to configure itself for 2*3TB+1*1TB disks which wasn’t the right configuration when the 4TB disk was fully used. To make it even more annoying the “btrfs filesystem resize” command takes a “devid” not a device name.

I think that when BTRFS is more stable it would be good to have the btrfs utility warn the user about such potential mistakes. When a replacement device is larger than the old one it will be very common to want to use that space. The btrfs utility could easily suggest the most likely “btrfs filesystem resize” to make things easier for the user.

In a disturbing coincidence a few days after replacing the first 3TB disk the other 3TB disk started giving read errors. So I replaced the second 3TB disk with a 4TB disk and removed the 1TB disk to give a 4TB RAID-1 array. This is when it would be handy to have the metadata duplication feature and copies= option of ZFS.

Ctree Corruption

2 weeks ago a basic workstation with a 120G SSD owned by a relative stopped booting, the most significant errors it gave were “BTRFS: log replay required on RO media” and “BTRFS: open_ctree failed”. The solution to this is to run the command “btrfs-zero-log”, but that initially didn’t work. I restored the system from a backup (which was 2 months old) and took the SSD home to work on it. A day later “btrfs-zero-log” worked correctly and I recovered all the data. Note that I didn’t even try mounting the filesystem in question read-write, I mounted it read-only to copy all the data off. While in theory the filesystem should have been OK I didn’t have a need to keep using it at that time (having already wiped the original device and restored from backup) and I don’t have confidence in BTRFS working correctly in that situation.

While it was nice to get all the data back it’s a concern when commands don’t operate consistently.

Debian and BTRFS

I was concerned when the Debian kernel team chose 3.16 as the kernel for Jessie (the next Debian release). Judging by the way development has been going I wasn’t confident that 3.16 would turn out to be stable enough for BTRFS. But 3.16 is working reasonably well on a number of systems so it seems that it’s likely to work well in practice.

But I’m still deploying more ZFS servers.

The Value of Anecdotal Evidence

When evaluating software based on reports from reliable sources (IE most readers will trust me to run systems well and only report genuine bugs) bad reports have a much higher weight than good reports. The fact that I’ve seen kernel 3.16 to work reasonably well on ~6 systems is nice but that doesn’t mean it will work well on thousands of other systems – although it does indicate that it will work well on more systems than some earlier Linux kernels which had common BTRFS failures.

But the annoyances I had with the 3TB array are repeatable and will annoy many other people. The ctree coruption problem MIGHT have been initially caused by a memory error (it’s a desktop machine without ECC RAM) but the recovery process was problematic and other users might expect problems in such situations.

Related posts:

  1. BTRFS Status March 2014 I’m currently using BTRFS on most systems that I can...
  2. BTRFS Status April 2014 Since my blog post about BTRFS in March [1] not...
  3. BTRFS Status July 2014 My last BTRFS status report was in April [1], it...

Andrew Pollock: [life] Day 309: Seaworld

Fri, 2014-12-05 15:26

My friends Alec and Christine were holidaying on the Gold Coast with their daughter Eloise this week, and I'd made plans to rendezvous with them at Seaworld for the day.

Sarah dropped Zoe around in the morning, and we departed not long after, with a side visit to the post office to clear my post office box.

We arrived at Seaworld just after 10am, and Alec and Christine were already inside and had secured some seats for the Seal Detective show. They saved us some seats and we joined them.

After that, we tried to watch the dolphin show, but they had to cancel it because someone had managed to trip over a stroller that was past the stroller parking area and had injured themselves. It seemed like quite the incident management, with a lot of Seaworld people descending on the scene, and erecting screens around the injured person. I have no idea why they weren't able to quickly move them and proceed with the show.

We grabbed some lunch and had the obligatory ride on the Vikings Revenge Flume Ride (that thing is such a Seaworld institution). It was getting hot, and Eloise was a bit tired and cranky from a full week of vacationing, so we decided to go back to their hotel and have a swim in the pool.

Zoe wanted to take the bus back with them, so I drove back to the Marriott and availed myself of their free valet parking, and we had a swim. The Surfers' Paradise Marriott certainly has a very impressive swimming area. We had lots of fun.

We ended up staying for dinner before heading back. Zoe fell asleep in the car on the way back despite my best efforts to prevent it. She had quite a meltdown when I had to get her out of the car when we got home, so I had to basically get her into her pajamas and pour her into bed and she fell back asleep easily.

It was a really nice day out.

Binh Nguyen: Music Production and Experimentation - Part 2

Fri, 2014-12-05 10:32
A follow on from:

http://dtbnguyen.blogspot.com/2014/04/music-production-and-experimentation.html

I've recently been working on music as an alternative hobby/career as has been indicated by the contents of this blog.



For an idea of some of the stuff that I'm interested in, please checkout my YouTube profile. There may be some remixes and some original stuff coming out in the near future via this avenue (some of the lower quality productions/samples are likely to go out via YouTube while the higher quality stuff is likely to go out via other outlets).

https://www.youtube.com/channel/UCwVJG67iHHPbmBxuHVbyOlw/playlists

https://www.youtube.com/channel/UCwVJG67iHHPbmBxuHVbyOlw/feed 

Compression as opposed limiting to reduce impact of increasing volume and limiting effect.

http://www.dummies.com/how-to/content/dynamic-processors-compressorslimiters.html

http://en.wikipedia.org/wiki/Dynamic_range_compression

http://www.audio-production-tips.com/limiter.html

http://www.soundonsound.com/sos/jun99/articles/mixcomp.htm



Some interesting information on creating Vinyl effects in Ableton.

http://www.musicsoftwaretraining.com/blog/how-to-create-a-vinyl-effect-in-ableton/

https://forum.ableton.com/viewtopic.php?t=2463

http://www.afrodjmac.com/blog/2013/09/21/free-ableton-live-pack-91-vinyl-scratcher-effect-plus-delay-modes-tutorial

http://www.afrodjmac.com/blog/2012/05/09/ableton-live-pack-56-vinyl-crackler

http://www.synthtopia.com/content/2012/05/11/ableton-live-vinyl-crackler-rack/



If you've ever heard electronic music you'll have heard strange electronic, vocal sounds before.

http://en.wikipedia.org/wiki/Vocoder/

https://www.gearslutz.com/board/electronic-music-instruments-electronic-music-production/567980-what-best-sounding-software-vocoder-if-such-thing-exists.html

http://manual.audacityteam.org/o/man/tutorial_vocal_removal_and_isolation.html

http://wiki.audacityteam.org/wiki/Vocal_Removal_Plug-ins

http://audionamix.com/



Listen closely to a lot of modern music and you'll realise that a lot of it contains multiple drum machine synthesisers with often clever overlapping and layering.

http://en.wikipedia.org/wiki/Drum_machine

http://www.attackmagazine.com/features/top-ten-classic-drum-machines/

http://www.sweetwater.com/c643--Drum_Machines_Modules

http://electronicmusic.wikia.com/wiki/Drum_synthesizer



Despite what I had previously thought about building modern electronic music it's not that easy.

http://www.wikihow.com/Write-a-House-Track

http://news.beatport.com/how-to-write-a-hit-house-track-10-top-tips/

http://www.musicradar.com/tuition/tech/26-house-production-tips-486841/

http://www.edmproducer.com/writing-melodies/

https://www.samplemagic.com/details/72/the-secrets-of-house-music-production

http://en.wikipedia.org/wiki/Song_structure_%28popular_music%29

http://en.wikipedia.org/wiki/Thirty-two-bar_form

http://www.wikihow.com/Produce-and-Write-Dance-Music

http://emusictips.com/2008/08/top-10-signs-your-electronic-music-is-amateur/

http://www.howmusicworks.org/910/Writing-Songs/Song-Structure

http://www.sommercamp.at/workshop2010/Songwriter/Writing_files/craftofSongwriting.pdf

http://dougashdown.com/An%20Introduction%20To%20Songwriting.pdf

http://www.songwritingfever.com/books/getstarted.pdf

https://garyewer.wordpress.com/2012/07/17/8-tips-for-writing-a-song-bridge/

http://homerecording.com/bbs/general-discussions/singing-vocals/how-do-you-write-bridge-song-structure-general-152631/

http://www.learneverythingabout.com/lyrics/song-structure-bridges.html

http://www.wikihow.com/Write-a-Bridge-for-a-Song

http://www.satoshitomiie.com/?p=766

http://www.learneverythingabout.com/lyrics/song-structure-bridges.html

http://www.soundstosample.com/blog/pro-tips/progressive-house-production-tips



Building software synthesisers for profit using Reaktor and other software.

http://basicsynth.com/

https://www.dubstepforum.com/forum/viewtopic.php?t=63293

https://www.udemy.com/build-a-synth-in-reaktor/

http://software-tips.wonderhowto.com/how-to/build-basic-modular-synth-reaktor-5-1-295767/

http://www.sonicacademy.com/Training+Videos/Course+Overview/Learn-How-To-Make-a-Synth-using-Native-Intruments-Reaktor.cid2095

http://www.synthtopia.com/content/2009/02/26/very-basic-modular-synth-made-in-reaktor/

http://www.nireaktor.com/reaktor-tutorials/building-a-basic-reaktor-synth/

http://www.synthedit.com/



Some notes that should keep your stuff from sounding amateurish.

http://www.nonalignmentpact.com/2008/09/telltale-signs-that-youre-amateur.html

http://www.hometracked.com/2008/02/05/auto-tune-abuse-in-pop-music-10-examples/



Sound effects feature heavily in modern music. Keep up with your reading.

http://en.wikipedia.org/wiki/Phasing_%28music%29

http://en.wikipedia.org/wiki/Phaser_%28effect%29

http://en.wikipedia.org/wiki/Bitcrusher

http://electronicmusic.wikia.com/wiki/Bit_crushing

http://en.wikipedia.org/wiki/Dither

http://en.wikipedia.org/wiki/Chorus_effect

http://www.sae.edu/reference_material/audio/pages/fullindex.htm



Some interesting software synthesisers/VST's out there.

http://boscomac.free.fr/

http://www.phuturetone.com/

http://www.vintagesynth.com/

http://freesoftsynth.com/

http://en.wikipedia.org/wiki/Software_synthesizer

http://www.musictech.net/2013/09/software-synths/

David Rowe: SM1000 Part 8 – Video and Beta CAD work

Fri, 2014-12-05 08:30

Over the last few months Rick KA8BMA has been working steadily on the schematic, PCB, and enclosure CAD work for the SM1000 Beta. This is now complete, the Beta PCBs have been made, and the first 2 Beta units are being hand assembled by Edwin at Dragino. Rick and I will test these, then kick off the Qty 100 Beta run which I estimate will be on sale in January.

I’m a bit intimidated by the mechanical CAD work required to make an enclosure, but Rick has stepped right up and done a fine job designing a bent metal case comprised of two U-shaped halves. This can be economically made in small quantities. This sort of case is more suited to the “small box on the desk” form factor, but that’s OK for now. It still has an aperture for the internal microphone. Click for larger images.







Last week I visited VK2 and gave a talk at the ARNSW Home Brew Group meeting at Dural, on the NW edge of Sydney. I had a very pleasant day with Peter Marx who was kind enough to drive me there and record the talk. As well as the standard intro to Codec 2 and FreeDV the second half covers the SM1000.