Planet Linux Australia

Syndicate content
Planet Linux Australia -
Updated: 13 min 45 sec ago

Erik de Castro Lopo: Forgive me Curry and Howard for I have Sinned.

Mon, 2015-11-16 22:22

Forgive me Curry and Howard for I have sinned.

For the last several weeks, I have been writing C++ code. I've been doing some experimentation in the area of real-time audio Digital Signal Processing experiments, C++ actually is better than Haskell.

Haskell is simply not a good fit here because I need:

  • To be able to guarantee (by inspection) that there is zero memory allocation/de-allocation in the real-time inner processing loop.
  • Things like IIR filters are inherently stateful, with their internal state being updated on every input sample.

There is however one good thing about coding C++; I am constantly reminded of all the sage advice about C++ I got from my friend Peter Miller who passed away a bit over a year ago.

Here is an example of the code I'm writing:

class iir2_base { public : // An abstract base class for 2nd order IIR filters. iir2_base () ; // Virtual destructor does nothing. virtual ~iir2_base () { } inline double process (double in) { unsigned minus2 = (minus1 + 1) & 1 ; double out = b0 * in + b1 * x [minus1] + b2 * x [minus2] - a1 * y [minus1] - a2 * y [minus2] ; minus1 = minus2 ; x [minus1] = in ; y [minus1] = out ; return out ; } protected : // iir2_base internal state (all statically allocated). double b0, b1, b2 ; double a1, a2 ; double x [2], y [2] ; unsigned minus1 ; private : // Disable copy constructor etc. iir2_base (const iir2_base &) ; iir2_base & operator = (const iir2_base &) ; } ;

Sam Watkins: sswam

Mon, 2015-11-16 13:29

TLDR: Division by zero is not as scary as it’s made out to be:

a/0 = b ⟺ a = 0

Division is multiplication, backwards. These two equations are exactly equivalent, by definition:

a/c = b

a = b×c

It’s easy to understand division by zero if we look at the equivalent multiplication.

a/0 = b

a = b×0

For any real number b:

a = b×0 = 0

a = 0

There are two cases with division by zero:

If a = 0, then a/0 = b is unconstrained, any real number b satisfies the equation. You can discard such an equation which does not constraint the result.

If a ≠ 0 then a/0 = b is contradictory. There is no real number b which satisfies that equation. This is still useful to know; “there is no answer” can be a sort of meta-answer. For example if trying to solve a system of equations of static forces, “there is no answer” might mean you need to consider a different design for your bridge!

There is no need to consider advanced concepts such as limits in order to fully understand division.

In short, a/0 = b is true if and only if a = 0.

If you see such an equation a/0 = b, you may simplify it to a = 0.

a/0 = b ⟺ a = b×0 ⟺ a = 0

a/0 = b ⟺ a = 0

I posted this here about a year ago:

David Rowe: Give Us Our Daily Bread

Mon, 2015-11-16 10:29

Last week I visited a modern Australian farm on the Eyre Peninsula of South Australia, about 500km from where I live in Adelaide.

This farm has been in one family for several generations, and has steadily grown to 8000 acres (3200 hectares). This same area was previously farmed by 7 families, and now provides a livelihood for just one. This tells me that modern agriculture is super efficient, and explains why food (and calories) are super cheap for us here in the affluent Western world.

This is both good and bad. Given the right political conditions, science and technology enables us to feed the world. We don’t need to be hungry and can use those excess calories for other purposes. The jobs lost in one industry migrate to others. This farming family, for example, has spawned a variety of professionals that have left the family farm and done good things for the world.

It also brings diseases of affluence. Our poor bodies are not evolved to deal with an excess of food. We are evolved to be hunter-gatherers, constantly on the look out for the next calorie. Historically we haven’t had enough. So we are hard wired to eat too much. Hence the rise of heart disease and diabetes.

Breathtaking Array of Skills

I was impressed by the diverse array of skills required to run the farm. Business, animal husbandry, mechanical, agricultural science. The increased mechanisation means computers everywhere and I imagine robotics is on the horizon. During our visit they were measuring the moisture content of the crop to determine the best time to harvest. They even have an animal “retirement village” – they care for several old working dogs who had kept foxes away from the sheep for years.

Unlike many jobs, they don’t know what their yields and hence income will be from year to year. That’s a lot of risk in your annual income.

Overall, It takes about 12 years to learn the skills needed to run a modern farm.

This farm produces 3,500 tonnes of wheat per year. Based on 13680 kJ/kg of wheat, and a person needing 8700 kJ/day, that’s enough to feed 15,000 people every year. From the work of one family farm. Wow.

Organic Farming

I asked them about organic farming. The bottom line is the yields would be halved. So double the prices for everything we eat. That may be fine if you are a rich Westerner but that is the line between life and death for someone in the developing world. Alternatively, it means using twice the land under cultivation for the same amount of food. Organic means starving poor people and goodbye rain forests.

Their use of pesticides is strictly monitored and all residues must be removed. They have modern, scientific methods of erosion control to manage the soil, and techniques to naturally fix nitrogen. Sustainability is being addressed right now by modern, scientific, farming.

In my opinion the organic food movement is a more about scientific illiteracy and marketing than health.

Wind Farming

On a nearby hill was a 75MW wind farm, part of many that have sprung up in South Australia over the past decade. I am quite proud that South Australia now averages 30% wind power. We are about to close down our last remaining coal power station.

In this case, the lucky farmer that owns the land leased for the wind turbines receives $100k per year in passive income. K-ching K-ching as the turbines rotate.

It’s incredible to think that for years there have been “rivers of energy” flowing over those hills. It took technology and the right economic conditions to reach up and pluck that energy out of the sky.

Michael Still: Mount Stranger one last time

Mon, 2015-11-16 07:27
This is the last walk in this series, which was just a pass through now that the rain has stopped to make sure that we hadn't left any markers or trash lying around after the Scout orienteering a week ago. This area has really grown on me -- I think most people stick to the path down by the river, whereas this whole area has nice terrain, plenty of gates through fences and is just fun to explore. I'm so lucky to have this so close to home.

Interactive map for this route.

Tags for this post: blog canberra bushwalk


Chris Smart: Btrfs RAID 6 on dm-crypt on Fedora 23

Sun, 2015-11-15 21:29

I’m building a NAS and given the spare drives I have at the moment, thought I’d have a play with Btrfs. Apparently RAID 6 is relatively safe now, so why not put it through its paces? As Btrfs doesn’t support encryption, I will need to build it on top of dm-crypt.

Boot drive:

  • /dev/sda

Data drives:

  • /dev/sdb
  • /dev/sdc
  • /dev/sdd
  • /dev/sde
  • /dev/sdf

I installed Fedora 23 Server onto /dev/sda and just went from there, opening a shell.

# Setup dm-crypt on each data drive

# and populate the crypttab file.

for x in b c d e f ; do

  cryptsetup luksFormat /dev/sd${x}

  UUID="$(cryptsetup luksUUID /dev/sd${x})"

  echo "luks-${UUID} UUID=${UUID} none" >> /etc/crypttab



# Rebuild the initial ramdisk with crypt support

echo "dracutmodules+=crypt" >> /etc/dracut.conf.d/crypt.conf

dracut -fv


# Verify that it now has my crypttab

lsinitrd /boot/initramfs-$(uname -r).img |grep crypttab


# Reboot and verify initramfs prompts to unlock the devices



# After boot, verify devices exist

ls -l /dev/mapper/luks*

OK, so now I have a bunch of encrypted disks, it’s time to put btrfs into action (note the label, btrfs_data):

# Get LUKS UUIDs and create btrfs raid filesystem

for x in b c d e f ; do

  DEVICES="${DEVICES} $(cryptsetup luksUUID /dev/sd${x}\

    |sed 's|^|/dev/mapper/luks-|g')"


mkfs.btrfs -L btrfs_data -m raid6 -d raid6 ${DEVICES}'

See all our current btrfs volumes:

btrfs fi show

Get the UUID of the filesystem so that we can create an entry in fstab, using the label we created before:

UUID=$(btrfs fi show btrfs_data |awk '{print $4}')

echo "UUID=${UUID} /mnt/btrfs_data btrfs noatime,subvolid=0 0 0"\

  >> /etc/fstab

Now, let’s create the mountpoint and mount the device:

mkdir /mnt/btrfs_data

mount -a

Check data usage:

btrfs filesystem df /mnt/btrfs_data/

This has mounted the root of the filesystem to /mnt/btrfs_data, however we can also create subvolumes. Let’s create one called “share” for shared network data:

btrfs subvolume create /mnt/btrfs_data/share

You can mount this specific volume directly, let’s add it to fstab:

echo "UUID=${UUID} /mnt/btrfs_share btrfs noatime,subvol=share 0 0"\

  >> /etc/fstab

mkdir /mnt/btrfs_share

mount -a

You can list and delete subvolumes:

btrfs subvolume list -p /mnt/btrfs_data/

btrfs subvolume delete /mnt/btrfs_data/share

Now I plugged in a few backup drives and started rsyncing a few TB across to the device. It seemed to work well!

There are lots of other things you can play with, like snapshots, compression, defragment, scrub (use checksums to repair corrupt data), rebalance (re-allocates blocks across devices) etc. You can even convert existing file systems with btrfs-convert command, and use rebalance to change the RAID level. Neat!

Then I thought I’d try the rebalance command just to see how that works with a RAID device. Given it’s a large device, I kicked it off and went to do something else. I returned to an unwakeable machine… hard-resetting, journalctl -b -1 told me this sad story:

Nov 14 06:03:42 localhost.localdomain kernel: ------------[ cut here ]------------

Nov 14 06:03:42 localhost.localdomain kernel: kernel BUG at fs/btrfs/extent-tree.c:1833!

Nov 14 06:03:42 localhost.localdomain kernel: invalid opcode: 0000 [#1] SMP

Nov 14 06:03:42 localhost.localdomain kernel: Modules linked in: fuse joydev synaptics_usb uas usb_storage rfcomm cmac nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 xt_conntrack ebtable_nat ebtab

Nov 14 06:03:42 localhost.localdomain kernel: snd_soc_core snd_hda_codec rfkill snd_compress snd_hda_core snd_pcm_dmaengine ac97_bus snd_hwdep snd_seq snd_seq_device snd_pcm mei_me dw_dmac i2c_designware_platform snd_timer snd_soc_sst_a

Nov 14 06:03:42 localhost.localdomain kernel: CPU: 0 PID: 6274 Comm: btrfs Not tainted 4.2.5-300.fc23.x86_64 #1

Nov 14 06:03:42 localhost.localdomain kernel: Hardware name: Gigabyte Technology Co., Ltd. Z97N-WIFI/Z97N-WIFI, BIOS F5 12/08/2014

Nov 14 06:03:42 localhost.localdomain kernel: task: ffff88006fd69d80 ti: ffff88000e344000 task.ti: ffff88000e344000

Nov 14 06:03:42 localhost.localdomain kernel: RIP: 0010:[] [] insert_inline_extent_backref+0xe7/0xf0 [btrfs]

Nov 14 06:03:42 localhost.localdomain kernel: RSP: 0018:ffff88000e3476a8 EFLAGS: 00010293

Nov 14 06:03:42 localhost.localdomain kernel: RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000

Nov 14 06:03:42 localhost.localdomain kernel: RDX: ffff880000000000 RSI: 0000000000000001 RDI: 0000000000000000

Nov 14 06:03:42 localhost.localdomain kernel: RBP: ffff88000e347728 R08: 0000000000004000 R09: ffff88000e3475a0

Nov 14 06:03:42 localhost.localdomain kernel: R10: 0000000000000000 R11: 0000000000000002 R12: ffff88021522f000

Nov 14 06:03:42 localhost.localdomain kernel: R13: ffff88013f868480 R14: 0000000000000000 R15: 0000000000000000

Nov 14 06:03:42 localhost.localdomain kernel: FS: 00007f66268a08c0(0000) GS:ffff88021fa00000(0000) knlGS:0000000000000000

Nov 14 06:03:42 localhost.localdomain kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033

Nov 14 06:03:42 localhost.localdomain kernel: CR2: 000055a79c7e6fd0 CR3: 00000000576ce000 CR4: 00000000001406f0

Nov 14 06:03:42 localhost.localdomain kernel: Stack:

Nov 14 06:03:42 localhost.localdomain kernel: 0000000000000000 0000000000000005 0000000000000001 0000000000000000

Nov 14 06:03:42 localhost.localdomain kernel: 0000000000000001 ffffffff81200176 0000000000270026 ffffffffa0925d4a

Nov 14 06:03:42 localhost.localdomain kernel: 0000000000002158 00000000a7c0ba4c ffff88021522d800 0000000000000000

Nov 14 06:03:42 localhost.localdomain kernel: Call Trace:

Nov 14 06:03:42 localhost.localdomain kernel: [] ? kmem_cache_alloc+0x1d6/0x210

Nov 14 06:03:42 localhost.localdomain kernel: [] ? btrfs_alloc_path+0x1a/0x20 [btrfs]

Nov 14 06:03:42 localhost.localdomain kernel: [] __btrfs_inc_extent_ref.isra.52+0xa9/0x270 [btrfs]

Nov 14 06:03:42 localhost.localdomain kernel: [] __btrfs_run_delayed_refs+0xc84/0x1080 [btrfs]

Nov 14 06:03:42 localhost.localdomain kernel: [] btrfs_run_delayed_refs.part.73+0x74/0x270 [btrfs]

Nov 14 06:03:42 localhost.localdomain kernel: [] ? btrfs_release_path+0x2b/0xa0 [btrfs]

Nov 14 06:03:42 localhost.localdomain kernel: [] btrfs_run_delayed_refs+0x15/0x20 [btrfs]

Nov 14 06:03:42 localhost.localdomain kernel: [] btrfs_commit_transaction+0x56/0xad0 [btrfs]

Nov 14 06:03:42 localhost.localdomain kernel: [] prepare_to_merge+0x1fe/0x210 [btrfs]

Nov 14 06:03:42 localhost.localdomain kernel: [] relocate_block_group+0x25e/0x6b0 [btrfs]

Nov 14 06:03:42 localhost.localdomain kernel: [] btrfs_relocate_block_group+0x1ca/0x2c0 [btrfs]

Nov 14 06:03:42 localhost.localdomain kernel: [] btrfs_relocate_chunk.isra.39+0x3e/0xb0 [btrfs]

Nov 14 06:03:42 localhost.localdomain kernel: [] btrfs_balance+0x9c4/0xf80 [btrfs]

Nov 14 06:03:42 localhost.localdomain kernel: [] btrfs_ioctl_balance+0x3c4/0x3d0 [btrfs]

Nov 14 06:03:42 localhost.localdomain kernel: [] btrfs_ioctl+0x541/0x2750 [btrfs]

Nov 14 06:03:42 localhost.localdomain kernel: [] ? lru_cache_add+0x1c/0x50

Nov 14 06:03:42 localhost.localdomain kernel: [] ? lru_cache_add_active_or_unevictable+0x32/0xd0

Nov 14 06:03:42 localhost.localdomain kernel: [] ? handle_mm_fault+0xc8a/0x17d0

Nov 14 06:03:42 localhost.localdomain kernel: [] ? cp_new_stat+0xb3/0x190

Nov 14 06:03:42 localhost.localdomain kernel: [] do_vfs_ioctl+0x295/0x470

Nov 14 06:03:42 localhost.localdomain kernel: [] ? selinux_file_ioctl+0x4d/0xc0

Nov 14 06:03:42 localhost.localdomain kernel: [] SyS_ioctl+0x79/0x90

Nov 14 06:03:42 localhost.localdomain kernel: [] ? do_page_fault+0x2f/0x80

Nov 14 06:03:42 localhost.localdomain kernel: [] entry_SYSCALL_64_fastpath+0x12/0x71

Nov 14 06:03:42 localhost.localdomain kernel: Code: 10 49 89 d9 48 8b 55 c0 4c 89 7c 24 10 4c 89 f1 4c 89 ee 4c 89 e7 89 44 24 08 48 8b 45 20 48 89 04 24 e8 5d d5 ff ff 31 c0 eb ac <0f> 0b e8 92 b7 76 e0 66 90 0f 1f 44 00 00 55 48 89 e5

Nov 14 06:03:42 localhost.localdomain kernel: RIP [] insert_inline_extent_backref+0xe7/0xf0 [btrfs]

Nov 14 06:03:42 localhost.localdomain kernel: RSP

Nov 14 06:03:42 localhost.localdomain kernel: ---[ end trace 63b75c57d2feac56 ]---


Looks like rebalance has a major bug at the moment. I did a search and others have the same problem, looks like I’m hitting this bug. I’ve reported it on Fedora Bugzilla.

Anyway, so I won’t do a rebalance at the moment, but other than that, btrfs seems pretty neat. I will make sure I keep my backups up-to-date though, just in case…

Steven Hanley: [mtb] Around the K 2013 - Cold morning and night lap of Kosci

Sun, 2015-11-15 11:25

The awesome open views heading toward Kiandra (fullsize)

Like the other Round the K galleries, another great day out on road bikes, this was the first time I had made it all the way around the loop too. The photo I am using to the left here is a great example of the open alpine regions neat Kiandra, those who have only done the Jindabyne - Cabramurra section have missed out on this bit of riding.

Gallery from the day is online Around The K 2012 gallery and as I said in the last few links to Round the K, bring on the next one in a few weeks. I am as this appears out competing in Triple Tri in pairs though so wrote the post ahead of time and am letting it appear during the day, unlikely that it matters as I doubt I have many readers.

And I have just noticed as I went to do an entry for Monday 2015-11-16 that I had in fact already posted the link and a photo for the post today. Oh well laziness is an artform so it is staying here.

Steven Hanley: [mtb] Blue Mountains Six foot/TNF100 scouting trip Feb 2014

Sat, 2015-11-14 22:25

Jane loving the trail run down to Coxs river (fullsize)

Oops I realised I forgot to link to this one in my reverse posting of all these adventures, this was a weekend Jane and I headed up to the Blue Mountains for some running and to scout out the Six Foot course (made Jane more comfortable on the course (and as she finished 2nd in the race it probably helped)) and for me we were able to do the climb up Furber Steps (and a nice run along Federal pass including the giant stair case descent).

Though I did get to climb the steps in the Mt Solitary Ultra I had not at that time planned to do that race so I was happy to see them for TNF100 prep. We had a good weekend up there and it was nice to have a relaxed run to the river and back, we managed to see a number of people out for a Fat Arse run on the course too. My gallery from my Blue Mountains weekend in Feb 2014 is online here, thanks for the company Jane, hope to see you back on the trails soon.

Francois Marier: How Tracking Protection works in Firefox

Sat, 2015-11-14 07:42

Firefox 42, which was released last week, introduced a new feature in its Private Browsing mode: tracking protection.

If you are interested in how this list is put together and then used in Firefox, this post is for you.

Safe Browsing lists

There are many possible ways to download URL lists to the browser and check against that list before loading anything. One of those is already implemented as part of our malware and phishing protection. It uses the Safe Browsing v2.2 protocol.

In a nutshell, the way that this works is that each URL on the block list is hashed (using SHA-256) and then that list of hashes is downloaded by Firefox and stored into a data structure on disk:

  • ~/.cache/mozilla/firefox/XXXX/safebrowsing/mozstd-track* on Linux
  • ~/Library/Caches/Firefox/Profiles/XXXX/safebrowsing/mozstd-track* on Mac
  • C:\Users\XXXX\AppData\Local\mozilla\firefox\profiles\XXXX\safebrowsing\mozstd-track* on Windows

This sbdbdump script can be used to extract the hashes contained in these files and will output something like this:

$ ~/sbdbdump/ -v . - Reading sbstore: mozstd-track-digest256 [mozstd-track-digest256] magic 1231AF3B Version 3 NumAddChunk: 1 NumSubChunk: 0 NumAddPrefix: 0 NumSubPrefix: 0 NumAddComplete: 1696 NumSubComplete: 0 [mozstd-track-digest256] AddChunks: 1445465225 [mozstd-track-digest256] SubChunks: ... [mozstd-track-digest256] addComplete[chunk:1445465225] e48768b0ce59561e5bc141a52061dd45524e75b66cad7d59dd92e4307625bdc5 ... [mozstd-track-digest256] MD5: 81a8becb0903de19351427b24921a772

The name of the blocklist being dumped here (mozstd-track-digest256) is set in the urlclassifier.trackingTable preference which you can find in about:config. The most important part of the output shown above is the addComplete line which contains a hash that we will see again in a later section.

List lookups

Once it's time to load a resource, Firefox hashes the URL, as well as a few variations of it, and then looks for it in the local lists.

If there's no match, then the load proceeds. If there's a match, then we do an additional check against a pairwise allowlist.

The pairwise allowlist (hardcoded in the urlclassifier.trackingWhitelistTable pref) is designed to encode what we call "entity relationships". The list groups related domains together for the purpose of checking whether a load is first or third party (e.g. and both belong to the same entity).

Entries on this list (named mozstd-trackwhite-digest256) look like this:

which translates to "if you're on the site, then don't block resources from

If there's a match on the second list, we don't block the load. It's only when we get a match on the first list and not the second one that we go ahead and cancel the network load.

If you visit our test page, you will see tracking protection in action with a shield icon in the URL bar. Opening the developer tool console will expose the URL of the resource that was blocked:

The resource at "" was blocked because tracking protection is enabled.

Creating the lists

The blocklist is created by Disconnect according to their definition of tracking.

The Disconnect list is on their Github page, but the copy we use in Firefox is the copy we have in our own repository. Similarly the Disconnect entity list is from here but our copy is in our repository. Should you wish to be notified of any changes to the lists, you can simply subscribe to this Atom feed.

To convert this JSON-formatted list into the binary format needed by the Safe Browsing code, we run a custom list generation script whenever the list changes on GitHub.

If you run that script locally using the same configuration as our server stack, you can see the conversion from the original list to the binary hashes.

Here's a sample entry from the mozstd-track-digest256.log file:

[m] >> [canonicalized] [hash] e48768b0ce59561e5bc141a52061dd45524e75b66cad7d59dd92e4307625bdc5

and one from mozstd-trackwhite-digest256.log:

[entity] Twitter >> (canonicalized), hash a8e9e3456f46dbe49551c7da3860f64393d8f9d96f42b5ae86927722467577df

This in combination with the sbdbdump script mentioned earlier, will allow you to audit the contents of the local lists.

Serving the lists

The way that the binary lists are served to Firefox is through a custom server component written by Mozilla: shavar.

Every hour, Firefox requests updates from If new data is available, then the whole list is downloaded again. Otherwise, all it receives in return is an empty 204 response.

Should you want to play with it and run your own server, follow the installation instructions and then go into about:config to change these preferences to point to your own instance:

browser.trackingprotection.gethashURL browser.trackingprotection.updateURL

Note that on Firefox 43 and later, these prefs have been renamed to:

browser.safebrowsing.provider.mozilla.gethashURL browser.safebrowsing.provider.mozilla.updateURL Learn more

If you want to learn more about how tracking protection works in Firefox, you can find all of the technical details on the Mozilla wiki or you can ask questions on our mailing list.

Thanks to Tanvi Vyas for reviewing a draft of this post.

Binh Nguyen: China Background, Economic Warfare, and More

Sat, 2015-11-14 00:56
- the world feels very different when you get perspectives from all over the world... if you were to simply watch the local news you'd think that the Chinese and Russians were right on our border and were ready to invade us. The other problem is that due to the language problem we only get a snippet of what they intend to say. Younger people in China aren't much different from us and censorship is bad but isn't as horrible as we're meant to believe?

Freedom, Politics and Change in China - Does The West Fear China Documentary

BBC Documentary Our World Flashpoint South China Sea english subtitles

United States, China and Public Opinion

Are We Looking For A Fight In The South China Sea

Are China's ambitions in the South China Sea a threat

The Debate - South China Sea Tensions (May 30th)

Counting the Cost - The scramble for the South China Sea

Taiwan in the South China Sea

Chinese Assertiveness in the South China Sea - Harbinger of Things to Come

Five Former U.S. Ambassadors to China Discuss U.S.-China Relations 

- if you listen to a lot of the what is being said it's a combination of fear, disbelief, concern, anger, etc... in varying quantities. A lot of countries are wanting to maintain current order or at least have an understanding of where they will fit into the world that is currently being shaped before our very eyes. Others wanting to change and looking for an idea of how far they can push things. There are a lot of commentators out there who have a limited understanding of the history behind what is happening, a lot of differing perspectives, highly concentrated media, on all sides, which makes it difficult to get a balanced idea of what is actually happening

The Heat - Henry Kissinger on China-U.S. relations

China and the U.S. Are Long-term Enemies-kd

India’s World – US-China face-off in South China sea

- the thing I find most bemusing is that people most often remember the most extreme examples of each and every society out there. If you were to listen to some media outlets it seems as though the Chinese government were against 'Falun Gung', 'Dalai Lama', etc.. for no reason. Dig further and most groups that the government is opposed to are wanting substantial social change (not judging here. There have been some pretty ugly accusations though...). The worse part of this is that while there is somewhat of a tacit agreement among intelligence agencies internationally on what type of covers/operations that they should and should not use. This may have changed of late somewhat with some targets/penetrations being considered of higher priority. Muddies the water a lot...

Kevin Rudd - Are China and the US doomed to conflict

The Debate - South China Sea Tensions (October 28th)

- they don't trust us and we don't trust them. Look at their history and you sort of understand why exactly they don't trust us. A lot of promises were broken. Since they have a long memory they're thinking why should they trust us if we can't be trusted to follow through on what we say. Makes the circumstances worse...

- if they want a 'peaceful rise' they'll need to export their culture either way. Make them seem less threatening and help us understand them within context. Whether it's the Russians, Chinese, Europeans, Arabs, etc... everytime they speak about US/Allied conspiracies they sound crazy. Music such as C-Pop, sporting/music stars, etc... help but they aren't accessible enough. Clearly, Putin believes in the old Soviet model of strengh entailing respect on the global stage whether in sport, technology, science, etc... Too simplistic. Fear and respect won't hold without a continual presence (similar to geo-political engineering). Admiration and respect is something different though. That holds and won't require a massive security apparatus to keep everything in order. Easier said than done with a lof of the problems the world faces now though. Something which China seems to be better at especially in the context of their neighbours...

- if you follow the financial markets you'll realise that a lot of things aren't adding up at the moment. A lot of numbers don't quite make sense. Others have noticed as well... not just the conspiracy theorists, speculators, etc... If we were to go on fundamentals many countries that technically shouldn't be in trouble are in trouble and vice-versa

$100 Trillion American Economic Collapse with Jim Rickards

Exclusive Interview - Jim Rickards & Peter Schiff Discuss Global Gold Markets [Full Discussion]

The Coming Financial Collapse Of Great Britain UK Explained _ Revolutionary Documentaries

China Warns US, It Has Begun Dumping Treasuries - Episode 751a

- one of the things that is obvious is that during moments of financial difficulty the US goes understands together and in solidarity for one another. Their debt purchases are split internally and externally. Think about the recent European Debt issues where interest rates when through the roof. By having someone step in and control the flow into the general community they've been able to manage interest rates, inflation, growth, etc... The US has made things slightly easier by having private entities step in to keep things in check. By using a proxy/third party it makes it more difficult for speculators if this is is what happening which would make it more difficult for the US. Who knows how much of their own debt they're actually buying?

- if the West goes to war it will be a multi-layered/complex war. Most countries that it is likely to go to conflict with have taken substantial measures to shield themselves from any impact that they likely to face. It will be economic, cyber, hybrid, conventional, and non-conventional warfare. With the way the US is being dealt with at the moment it feels as though it's enemies have found a moment of weakness (or else the US is in actual decline). They're basically seeing how far they can push the US and it's allies at the moment. The obvious question is how much will it weaken and whether or not it will be (relatively) terminal?

Cold War 2 Or World War 3 Economic Warfare Between The United States And Russia Has Begun

Cyber-Enabled Economic Warfare - An Evolving Challenge

- the West is getting outplayed. If you want to take a bet, there are plenty of under valued assets out there if you look hard/deep enough. Moreover, it's clear that prior to any major military move that is likely to trigger actions by others (such as sanctions) a lot of countries are betting on this and taking a bet on it to reduce their economic impact. In reality the US has been caught off guard a number of times... but it doesn't really matter if you have a massive military. Does it matter if you can't really afford (or have no appetite) to deploy it?

The Push For War With China Is Now Escalating -- Episode 234

Economic WAR Between U.S. & Russia _ Gregory Mannarino

Panel 2 - Russia, China, and the Future of  Economic Warfare

- if various parties have engaged in economic/algorithmic based warfare then it would explain a few odd market movements and why some people have been arrested for reasonably 'normal behaviour' (according to the press). Part of me feels as though the world is currently being re-shaped in front of us behind our backs (if that makes sense)

- the problem with a lot of activists and conspiracy theorists is that they sound crazy or that they mix up good with bad material. It makes it very difficult to judge their credibility. This is especially the case with financial makret speculators who have a bet on the other side

US Pushes War Against Russia, North Korea And China To Cover-Up The Collapse - Episode 747b

U.S. Government Financial Numbers Are Manipulated To Keep The Illusion Of A Recovery - Episode 786a countries are worried about surprise opportunistic moves. Think about Turkey, Iran, Saudi Arabia, Lebanon, Iraq, Israel, etc... in the Middle East region at the moment. With the advent of the Syrian/Iraq/Afghanistan conflicts all have made moves to re-shape the region as they see fit. US/Australia has troops stationed in the north in case an 'opportunity' rises (there are other reasons as well obviously)...

- difficult to read encoded URL's. Thankfully, don't need to memorise them (though you do tend to memorise things over time)...

- making extensions easier then you actually think

- Carla is a sound plugin host for Linux. May require code modification/re-compilation to get things running on your localhost lots of plugins to help secure your local setup...

- Ninite is not the only option for automated installation under Windows. If desperate plenty of options

- decoding video/sound streams used to be easy but is not becoming more difficult with the increase in encoding, obfucation, encryption, etc... Need more time but think I can come up with an elegant solution... (some of the existing code that I'm looking at is highly specific and needs extensive modification for each site. I want a generalised solution that is elegant if possible...)

Some recent interesting quotes in the media...

- Multiple Air Force and industry sources confirm that the Raptor has a lower radar cross section over a wider range of frequencies than the F-35 (as the Air Force maintained for nearly decade till 2014), but the newer aircraft is far better at managing its signature thanks to an incredibly advanced electronic warfare suite. That is likely why retired Air Combat Command commander Gen. Mike Hostage told Breaking Defense: “The F-35 doesn’t have the altitude, doesn’t have the speed [of the F-22], but it can beat the F-22 in stealth.” The operative word there is can. As current ACC commander Gen. Hawk Carlisle told National Defense Magazine: The F-35 has much better “passive capability to determine who’s out there [and] its ability to manage its own signature.”

Ultimately, it’s the pilot vehicle interface the United States has developed over the decades at great expense that affords it the edge over Russia and China’s upstart programs—as Carlisle himself told me a few years ago at the Pentagon. Nonetheless, the United States will have to keep developing new technology to stay ahead.

- "If the only problem the F-35 had was that the aircraft was $1 million more expensive, they wouldn't have a problem," he said. "The problem is the aircraft is tens of millions of dollars more than they originally told people it would be, and that's just the acquisition price. It's the sustainment cost that will destroy air forces."


Still, even with Canada pulling out of the program, costs of the F-35 will likely fall in the long term as production of the aircraft becomes more efficient, according to The Fiscal Times. Each plane now costs an estimated $108 million, according to Lockheed, and prices are expected to fall to $85 million per plane by 2019 if Canada stays in the program.

- The F-35 program includes variants for the Marine Corps, Air Force and Navy, and also has international developmental partners and customers. The fighter program has been plagued by numerous problems from helmet glitches that made pilots air sick to software issues. Most recently, the services discovered that pilots weighing less than 136 pounds could be killed by whiplash if they needed to eject

- Because of the importance and complexity of the project, Alexander Sergeyevich Yakovlev assigned a large portion of his OKB to the development of the new VTOL fighter, with no fewer than ten chief engineers working simultaneously on what was called "Product 48" (the military had designated it Yak-41). Over fifty designs were studied. One key problem was designing an aircraft with both vectoring thrust and an afterburner, which was essential for sustained supersonic speeds. A twin-engine design was considered, but abandoned as the loss of an engine on landing would result in an immediate roll to the side. Eventually it was decided that the best arrangement was a single vectoring nozzle located just behind the center of gravity, as well as dedicated vertical thrust jets positioned just behind the cockpit. A considerable amount of time was spent in the development of a flat, rectangular nozzle similar to that later employed on the American F-22 Raptor. Such a nozzle proved well-suited for the changes in configuration needed for both thrust vectoring and supersonic flight, and allowed for a thin, shallow tail. Ultimately, a circular nozzle was used, located between twin booms supporting the twin-finned tail.[1][page needed]

Parts subject to excessive heat from the engines during landing were manufactured of titanium, and no less than 26% of the overall aircraft was to be manufactured of graphite or composite material. Because of heat build-up, hovering was restricted to no more than 2½ minutes.[1][page needed]

All three engines were controlled through an interlinked digital system, which was capable of controlling both engine start-up as well as modulating the thrust of all three engines during landing and hovering flight. Twin tandem reaction control jets were positioned at the wingtips, while a swiveling yaw jet was positioned under the nose.[1][page needed]

The cockpit was pressurized and air-conditioned. The small canopy was bulletproof in front. It hinged to the right, but because of a long dorsal spine it had no rear vision. The ejection seat was automatically armed as soon as the engine duct was rotated past 30 degrees with an airspeed of less than 300 km/h (186 mph). The instrumentation in the prototypes was simple and similar to that planned for the earlier Yak-36M. The production version was to have been fitted with an extensive avionics and weapons suite including doppler radar, laser-TV ranging and aiming, as well as a heads-up multifunction display (HUD) which worked in connection with a helmet-mounted missile aiming system as found on the Mikoyan MiG-29. This system allows the pilot to lock onto an enemy aircraft by turning his head as far as 80 degrees from front.[1][page needed]


Following the announcement by the CIS on September 1991 that it could no longer fund development of the Yak-41M, Yakovlev entered into discussions with several foreign partners who could help fund the program. Lockheed Corporation, which was in the process of developing the X-35 for the US Joint Strike Fighter program, stepped forward, and with their assistance 48-2 was displayed at the Farnborough Airshow in September 1992. Yakovlev announced that they had reached an agreement with Lockheed for funds of $385 to $400 million for three new prototypes and an additional static test aircraft to test improvements in design and avionics. Planned modifications for the proposed Yak-41M included an increase in STOL weight to 21,500 kg (47,400 lb). One of the prototypes would have been a dual-control trainer. Though no longer flyable, both 48-2 and 48-3 were exhibited at the 1993 Moscow airshow. The partnership began in late 1991, though it was not publicly revealed by Yakovlev until 6 September 1992, and was not revealed by Lockheed until June 1994.[1]

- Iran is not an innocent country burdened by our sanctions, as some like to portray it. It is a country that deprives its citizens of basic needs in order to bankroll terrorism and violence throughout the world. Iran’s interests are far different than our own and to believe that handing over billions of dollars to this regime will go without bolstering our enemies is ludicrous. To ignore Iran’s intentions in the world is foolish. And to believe that this is a good deal is simply naïve.

- In Israel, much of the criticism has revolved around the cost of the US-made jet and the erosion of indigenous know-how. Former defense minister Moshe Arens, an aeronautical engineer by training and one of the program’s most vocal castigators, told The Times of Israel in October that while the F-35 might be “nice to have,” he didn’t see any need for it considering the country’s budgetary constraints. He noted that the military was still operating Vietnam War-era armored personnel carriers — to fatal effect this past summer in Gaza’s Shejaiya neighborhood this past summer — and said Israel would do better upgrading its existing F-15 and F-16 planes and investing the surplus funds in the ground forces.


In 1968, Israel bought the US-made Phantom, which was faster than the Mirage and could carry nearly six times its payload. “Our concept is that we will never win with quantity,” Lt. Col. B said. “We’ll win by being first.” The Phantom, he said, was “the first bomber that could escort itself deep into enemy territory.”

- This is not to say that today’s IAF planes lack the ability to unlock the S-300. Quite likely, the IAF has trained against the system in Greece and has created a combat doctrine capable of defeating it. The F-35 though, he said, “is similar to the iPhone,” in that the planners were able to take the capacity once housed on separate aircraft – stealth, intelligence gathering, advanced radars, planning, control, and electronic warfare – and “pack it all into a single fighter plane.”

Shapir conceded that the aircraft has “fantastic” capabilities and even said it might yet prove a useful tool against the S-300, but asserted that the only reason it is a truly necessary tool for Israel – which fights most of its battles near home but needs to maintain the capability of projecting its air power to places as distant as Tehran — is because Israel’s planes are aging and the United States “made the F-35 the only game in town.”

“There’s no other way,” he said, “because there’s nothing else out there.”

- Which raises the question of whether the RCAF will get new fighters at all. The lifetime of existing CF-18s has already been extended to 2025. The Liberals appear determined to end the RCAF’s participation in the aerial campaign against the Islamic State of Iraq and the Levant. Bearing that in mind, they may decide it makes more sense to invest in state-of-the-art drones, which can stay aloft virtually around the clock and patrol vast swathes of Arctic territory at high altitudes, than replace aging but still-serviceable manned fighters they would prefer not to use. In that event, there would savings in the billions, which could be redirected towards a navy in dire need of rapid, major investment.

- Because of their relatively long wavelength, VHF radars generally lack sufficient accuracy to guide a missile to a target on their own and are therefore used to cue higher frequency, shorter wavelength engagement radars to the approximate location of the target. Narrowband stealth aircraft such as the F-117, F-22 and F-35 were designed to be very low observable (VLO) in these higher frequencies in order to significantly limit the range at which they can be successfully detected by engagement radars. Consequently, despite inputs from the VHF acquisition radar, the X-band* engagement radar of Dani’s SA-3 battery was able to track the F-117 only at a distance of 8 miles (13 km), obtaining a lock and launching two missiles towards it only on the third attempt (the colonel would order his men to switch the engagement radar on for no more than 20 seconds for each attempt in order to avoid being targeted by NATO electronic warfare aircraft).

- Meanwhile, Germany spends a mere 1.2 percent. Italy, Canada, and Spain spend 1 percent or less. It’s understandable that people in those countries prefer to spend their money on universal health care and paid parental leave. But one of the reasons they’re able to do that and skimp on defense is the security subsidy they get from US taxpayers. The United States foots the bill for 73 percent of NATO’s defense spending, including the cost of keeping more than 40,000 troops in Germany. The fact that so many Europeans have come to take US protection for granted could be seen as a sign of the trust they place in the US-led NATO alliance. But a truly strong alliance requires equal participation from all members. Europeans can’t expect Americans to make sacrifices to defend them if they aren’t willing to make the same sacrifices to defend themselves.

- The humiliating failure of the two peace agreements signed in Minsk, Belarus, intended to halt the fighting in eastern Ukraine, proved what leaders of the free world simply refuse to admit: that there is no dealing with Putin the way they deal with one another. The model is repeating itself in Syria, as diplomats head to Vienna for peace talks. But confronting Putin doesn’t mean defeating the entire Russian army or starting World War III. Putin’s entire leadership cult in Russia is built on his image as an invincible strongman. He cannot afford to look like a loser, which is why he has maintained the feeble myth that Russian forces aren’t fighting in Ukraine, and why he picks targets NATO won’t defend. Any opposing force that threatened to inflict enough damage to pierce Putin’s illusion of invincibility would be enough to cause a real change in his behavior.

But the politicians of the free world know that it is easier and more popular to do nothing and claim to be peacemakers than to endure the criticism that inevitably comes with any action, which is why it will be so hard to break the cycle in Ukraine, Syria, and wherever Putin prods next—whether it’s Libya, the Baltics, or Venezuela. The United States and Europe have overwhelming military and economic advantages over Russia, but their leaders seem to lack the realization that diplomacy has its limits when facing dictators, and that diplomacy is only possible from a position of strength. As long as Putin sends jets and tanks while the West sends blankets and diplomats, the dictator will be calling the shots.

In 1986, Ames told the KGB that he feared he would be a suspect after the loss of several CIA assets. The KGB threw US investigators off his trail by constructing an elaborate diversion whereby a Soviet case officer told a CIA contact that the mole was stationed at Warrenton Training Center (WTC), a secret CIA communications facility in Virginia. US mole hunters investigated 90 employees at WTC for almost a year and came up with ten suspects, although the lead investigator noted that "there are so many problem personalities that no one stands out".[22][23]

- China's efforts amount to a worldwide "market intelligence program," says former FBI analyst Paul D. Moore. "The reality is that China does not practice intelligence the way God intended," he jokes. America's intelligence structure arose during the Cold War to contain the Soviet Union. "In our model, professional intelligence officers go out and do the job," Moore says. "In China's model, anyone and everyone is a potential intelligence asset."

- “Of course, we too practice cyberespionage,” Clapper said. “In a public forum, I won’t say how successful we are at it, but we’re not bad at it. When we talk about what are we going to do to counter espionage, to punish somebody, or retaliate, I at least think it’s a good idea to think about the old saw that people who live in glass houses shouldn’t throw rocks.”

That comment didn’t sit well with the committee’s chairman, Sen. John McCain, a Republican from Arizona.

“So it’s OK for them to steal our secrets that are most important, including our fighters, because we live in a glass house?” McCain asked. “That is astounding.”

Clapper replied, “I’m not saying it’s a good thing. I’m just saying that both nations engage in this.”

- “We should not have one-sided evaluations. People fell in love in the camps, people got pregnant; it wasn’t all bad,” he says, attributing negative information about the camps to a western campaign against Russia. “It was fashionable to say bad things about the USSR. Now it is again fashionable to insult Russia. We have sanctions against us. The west looks for negative things.”

Panikarov’s views on the Gulag are part of a larger trend. With the Soviet victory in the second world war elevated to a national rallying point under Vladimir Putin’s presidency, the forced labour camps, through which millions of Soviet citizens passed, are seen by many as an unfortunate but necessary by-product. In many museums and in much public discourse, the Gulag is not ignored completely, but is “contextualised” in a way that plays down the horror and pairs it with the war, suggesting the two come as a package.

- "If you want to hit an aircraft carrier, you just drop a bomb on the flight deck, and that puts the carrier out of action," he said, saying flight deck incidents have caused many deadly carrier fires over the years.

"You get a weapon — the bigger the better — and put it on the flight deck, preferably when they're launching, recovering or arming aircraft," Polmar said. Or, he added, "You knock out the propellers" with a torpedo designed to home in on their movement.

- As for new technology, Rear Admiral Ma said China has tested a new launch system “many times” and that all tests so far have gone quite smoothly. Ma spoke of “breakthroughs” in an electromagnetic catapult launch system for the new carrier. The new technology will set it apart from the Liaoning, which uses a more-outdated “ski jump” launch system. Breakthroughs in developing a catapult system would result in an “enormous increase” in the flight radius and payload of carrier-based aircraft, Ma said. With this technology, Ma claimed, China will be on par with or even more advanced than the United States.

Ma would not confirm, however, that the new technology was being used on the carrier currently under construction. Admiral Liu said the new carrier would “definitely” have areas of improvement over the Liaoning but declined to provide any specifics, saying the construction process is “extremely complicated.”

- China and Germany agreed to work on stopping economic cyber spying between the two nations amid mounting concern that the thousands of small- and medium-sized companies that form the backbone of German industry are ill-equipped to repel hacking attacks.

Similar no-spy agreements exist between China and the U.S. as well as the U.K., Merkel said Thursday in Beijing. Germany, the Asian nation’s biggest European trade partner, seeks such a deal “very quickly,” and China agreed, she told reporters after talks with Chinese Premier Li Keqiang.

- Britain spends £37.4 billion on its military budget, the fifth largest in the world.

Of this, £19.5bn is with British industry but less than half of new contracts are put out to competitive tender.

BAE Systems, Britain’s largest manufacturer, is the main supplier. In 2014 only 8 per cent of its contracts with the MoD were competitive.

Over 60 per cent of British arms sales are to the war-torn Middle East. Since 1945, British forces have carried out armed intrusions in foreign countries on 25 occasions — more than any other nation, including the US and Russia. Syria awaits.

Over the past 25 years Britain has spent £34bn on such interventions, mainly in Iraq and Afghanistan, where the army suffered significant military defeats.

This sum increases to £42bn if compensation for injury and death is included, plus a further £30bn on long-term care for veterans.

The consequences for the people of those countries are now only too visible, with thousands of refugees leaving their homes to seek respite in Europe from bombing, shelling and starvation. "By keeping silent," said Mr Shlosberg, "Russia's rulers have shown how far away they are from the Russian people -- on such unreachable heights that they hear nothing, feel nothing and understand nothing. The landing of their aircraft will not be a soft one."

- Back in 2004, when Australia was in the process of negotiating a trade deal with the United States — one that John Howard initiated — we were told that there would be no changes to the Pharmaceutical Benefit Scheme, the great scheme that ensures that all Australians, no matter their economic status, will have access to medicines at a reasonable price.

The US pharmaceutical industry hates the PBS with a passion because it would love to get Australians to pay much higher prices than what we do. Howard knew that it would be political poison, especially with an election due in 2004, to say anything indicating that the Americans would be able to manipulate the PBS. But that is precisely what happened. Young children brought up not believing in God are more likely to be generous and tolerant than those who grow up in a religious household, a study has suggested.

Agnostic and atheist kids were significantly more likely to share than children whose parents were religious, researchers claimed.

But children who believe in God were more likely to be vengeful and back harsher punishments for those who hurt others, they said.

It is suggested this is because religious children feel as they are going to heaven they are less concerned about the consequences of being mean.

- “Yeah, the good old prosperous days when US had a country that cared about Israel and our own morality. We stopped communism dead in South America. Consider how things would be now if Obama had been running things then. Instead of an Islamic Spring, we’d have had a Commie Spring. Mexico under communist rule, our borders being overrun worse than they are now, was a viable possibility back then. Cuba would’ve been thrilled. Even Jimmy Carter would have been happy.”

What a load of crap. America back in the 80s faced very different threats than today, and even Reagan didn’t let Israel control our foreign policy in the Middle East. Imagine where we would be now if Alfred E. Bush hadn’t invaded Iraq, or enacted his economic policies that cratered our economy. Whichever President that took over after GW would have had to make similar choices to what current administration has done. We are simply tapped out economically and militarily to repeat what was done during the Cold War, let alone shoving our weight around the world. Grow up.

- A central thrust of Soviet propaganda throughout the Cold War was to portray all Soviet misconduct, however outrageous, as no different from what the West was doing — including the propaganda itself. Accordingly, if the West accused the Kremlin of some gross wrongdoing, it was promptly depicted as another hypocritical attempt to belittle the Soviet Union. To a degree it worked: Many Russians, lacking any direct experience of the West, accepted a moral equivalence between their system and Western democracy — along with an instinctive fear of a world forever scheming against them. Alas, this approach has become an integral part of Vladimir Putin’s Russia.

Steven Hanley: [mtb] Happy Jacks and Jagungal wilderness ride on NYE 2012

Fri, 2015-11-13 14:25

David, Julie and Alex with Jagungal in the background (fullsize)

Alex and I had been thinking we should head up to near Jagungal and check out some of the trails through the wilderness there to ensure bikes could get through. This to ensure our planned route through for the next Canberra to Kosci Ride would work better than the previous one.

We decided to do a new years eve mtb ride on our single speeds in the wilderness around Mt Jagungal (the northern most mountain in Australia over 2000 metres). David and Julie came along for the fun, though I did not have my camera I was able to get my phone out easily through the day and get some good photos. They are all online in my Happy Jacks Jagungal Ride Album. Nice day out and and awesome way to finish off the year even though we arrived back in canberra tired and ready for sleep around midnight.

Steven Hanley: [mtb] ARNuts Victorian Cycling Holiday 2013

Thu, 2015-11-12 12:25

The boys on the trip overlooking Halls Gap (fullsize)

As I say i the write up, late in 2012 Alex, Lib and I started plotting a week of cycling in Victoria. Inspired to some extent by a cycling tips article, we added in mtb riding to the plan and decided to ask Bleeksie, Brooke and Aaron along for the fun.

We klicked it off with the Otway Odyssey mtb race and then did a bunch of awwesome mtb and road rides aorund the state. I was rather impressed with the grampians having never been there. Such a great week on bikes.

Photos and a few words from the trip are online in the gallery Victorian Epic Cycling Holiday February 2013. Now we just need to work out the next such trip. I suspect a south east Queensland cycling trip could be the go for some time in the future. I know there is a bunch of great riding up there.

Steven Hanley: [mtb] Whungee Wheengee Canyoning

Wed, 2015-11-11 13:25

A more open section later in the canyoning day (fullsize)

The previous ARNuts canyoning trip had been based on Mark's birthday, this time we all got up there to celebrate Alex's birthday. Another fun day out in the Blue Mountains with the ARNuts.

My photos from the day are on my Whungee Wheengee Canyoning page. I should not find it so amusing but I do that when the guides end up with a group of people who have fun in long AR events or 100KM runs and similar it seems to be a bit of a shock to them as they are used to clients that struggle with the hikes and harder bits rather than clients who are fitter than most people and simply get on with the harder bits having more fun the harder it gets.

Colin Charles: Ubuntu Online Summit: MySQL & Variants in 16.04

Tue, 2015-11-10 17:25

I personally have always enjoyed the Ubuntu Developer Summits (UDS), but nowadays they have been converted to the Ubuntu Online Summits (UOS). Attending them is not always convenient (timezone issues, might be travelling, etc.) so I watched the recorded video of a session I was interested in: MySQL & Variants in 16.04.

My key takeaways
  1. Ubuntu 16.04 Xenial Xerus is an LTS release.
  2. The term “cross-grade” is used a lot (it is not about downgrading/upgrading, but being able to use MySQL or MariaDB or Percona Server interchangeably)
  3. It would be nice to see MySQL 5.7 in this release (for Xenial as well as Debian Stretch). From Oracle there is a new packager taking over the task (Lars)
  4. MySQL 5.5 is still the default in Debian, and there needs to be upgrades tested between 5.5 to 5.7 (it looks like the ideal jump is that Ubuntu will not be seeing MySQL 5.6)
  5. Percona Server 5.7 is 60-90 days out; xtrabackup has had some new modifications and deserves an upgrade
  6. Boost is a new requirement for MySQL 5.7 & Percona Server 5.7; some old TokuDB problems in the builds are likely already fixed in MariaDB Server so this can be inherited
  7. MariaDB is waiting to iron out the bugs in 10.0, and may stick to that
My “raw” transcribed notes
  • Attendees:

    • Jon Grimm (Engineering Director for Ubuntu)
    • Robie Basak (Ubuntu)
    • Otto Kekäläinen (MariaDB Foundation)
    • Lars Tangvald, Norvald H. Ryeng (Oracle)
    • George Ormond Lorch III (Percona)
  • Robie: Waiting in Debian for a transition slot from MySQL 5.5 to MySQL 5.6. There’s some discussion with bugs, re: Akonadi, need to also resolve ABI issues with MySQL 5.6. Not really discussed MySQL 5.7 yet.

  • Norvald: 5.7, changes to installation. Client library ABI cleaned up. There may be some clients breaking because of that. No more exported symbols. See: The Client Library, Part 1: The API, the Whole API and Nothing but the API & The Client Library, Part 2: The Version Number
  • mysql_install_db is now replaced by --initialize in the server, so have to rewrite the post-install scripts. Might also have some AppArmour changes. Spoke to people @ DebConf (so best place is to put AppArmour profiles upstream (i.e. in mysql) and Debian and other distros will get it from there). AppArmour profile is in the MySQL source package now. Probably can get away with doing everything as cmake variables.
  • MySQL 5.7 has disabled the old password hashing algorithm, so if people haven’t upgraded they might have problems; so a manual intervention to fix their accounts.
  • Going from MySQL 5.7 to MySQL 5.6? It is done by dump and restore. There is no testing automated downgrades. Are there disk format changes? Norvald is not aware of any. If you use virtual columns in 5.7, you can’t downgrade easily to 5.6.
  • Robie would prefer to not release 5.6 and 5.7 concurrently. During Trusty, there was some level of user confusion. Debian – release team would prefer to see one transfer than two, so is it better to just do a single transition to 5.7?
  • Norvald says there hasn’t been testing from 5.5 -> 5.7. They only support upgrades from 5.5 -> 5.6 -> 5.7. For Ubuntu the choice can be to have 5.6 and then later do 5.7, but Jessie only just released with 5.5, so Stretch with 5.6 might not be a great idea (so users migrating from Jessie to Stretch will go from 5.5 to 5.7). Could also have 5.7 depend on a stripped 5.6 binary (like the embedded server; this is for localhost and the security team shouldn’t be too annoyed) for people to do an upgrade. Norvald says this has not been tried and there needs to be a migration path tested from 5.5 -> 5.7.
  • Conclusion: 5.7 in Stretch. Xenial is an LTS release, and 5.7 should be targeted for that.
  • If the maintainer script fails (postinstall script fails – don’t leave apt in a weird state). If it fails then upgrades, leave a debconf critical notice to say that the service is disabled and then fix it manually. Otto says that leaving /etc in a broken state is terrible, so we should avoid it.
  • Do we (Oracle) have the resources for 5.7 packaging and how soon can it be done in time for Xenial? There were patches from Lars in the git tree, but there haven’t been more recently. Lars will take over the 5.7 transition so if there is a list of work items, this will be settled (Lars will take over from Norvald).
  • There will be a separate session with Norvald/Lars/Robie outside of UOS about 5.7. Defer the Boost conversation after the session as well.
  • George: Percona is mainly looking out towards the 5.7 work and what kind of resources that will be put to that. There are new folk @ Percona to help with this. Percona inherits so much from the upstream codebase, it just works for Percona Server. There is Percona XtraDB Cluster and Percona xtrabackup, and xtrabackup has moved on quite a bit since the last upload (since last November 2014). So might be good idea to look at a refresh. There has also been a lot of work done on Percona XtraDB Cluster and there are some developments with Codership, so they are unsure if they will have their own Percona XtraDB Cluster 5.7 by the time Ubuntu is supposed to ship. When Percona is ready for something, just give Robie a shout to ensure that things happen. 60-90 days before a Percona Server 5.7 release. Just be aware of feature freeze for Xenial.
  • Norvald mentions that Percona Server 5.7 will also depend on Boost and there needs to be a decision on this. George mentions that TokuDB is now part of Percona Server, and it has some of its own requirements as well. Do we include TokuDB? It has requirements like it will only run on 64-bit platforms. Things to figure out going forward? MariaDB has been carrying TokuDB last November, but Robie remembers disabling it in Ubuntu. George says there were some licensing issues back then but they seem to be taken care of.
  • Otto says the builds for TokuDB was failing. It has a dependency on jemalloc, and that might have been the reason there were failures (says George). There may be something else where it doesn’t build on Ubuntu builders. But Otto says that there was a commit where this got fixed about last month. George will follow on, just to absorb it, since the legwork is already complete.
  • Otto: Trusty has 5.5, and Jessie and all other Ubuntu releases have 10.0, and 10.1 was released last month and I’m not quite pushing it to Debian quite yet. Fix 10.0 build fixes, upstream them, then only focus on 10.1. Blocking? (last summer) 5.6 is not in testing, so could not depend on it/changes done in 5.6 mysql-common. Here’s hoping that mysql-common going forward will be generated separately.
  • Robie will take an action to resolve the delta (probably just drop it). To sync MariaDB 10.0 to Xenial.
  • Discussion on /var/lib/mysql/*.flag thing on the list — conclusion at: mailing list — goal: within a single Ubuntu release, people can “cross-grade” between MySQL variants. The goal is to support all 3, and users want to try them, and thats when the bug reports come. Robie’s goal: move to a per-variant data directory. Otto says that once directory names change, 3rd party tools might have breakage. So a working prototype. Migration path is difficult. Maybe the best is to turn /var/lib/mysql into a symlink and store the data elsewhere. PostgreSQL does per version directories today; so studying that is going to happen.

Steven Hanley: [mtb/events] Sri Chinmoy Trail Ultra 2013 - 100 KM (first year)

Tue, 2015-11-10 16:25

At the start line (fullsize)

This was the first year Sri Chinmoy had run this Ultra, an event now in the third year (and I again ran it this year in a team and once more doing 2 legs). In 2013 they had not yet decided on the increase by 1km every year plan, however the course already showed off Canberra really well.

Alex and I decided to Alternate legs, in retrospect probably harder than doing 2 in a row each, however we had a good day out, and this year I got to do 2 in a row to compare. Of course I had my camera out there and took a bunch of photos which are online in my Sri Chinmoy 100 2013 gallery.

Steven Hanley: [mtb] Yell for Cadel, Australia's best ever XC mountain biker won the tour!

Tue, 2015-11-10 16:25
I know it has been a few weeks, however I have not exactly been on a blogging rampage, what with having my first ever month of no entries here. However I should start writing again and this is something of note for sure. How exciting it is that Cadel Evans won the tour!

I have been a fan of Cadel for a while, I guess since reading mtb magazines through out the 90s and marshaling at the mtb National Championship races in Majura Pines in Canberra when he won the title here. That he won the MTB World Cup series for two years in a row, has also won the Road World Cup series two years running, won the Road World Champs and now the Tour de France it is fairly obvious to all he is the most complete successful cyclist Australia has ever produced.

I still remember watching him lead through some of the single track at Majura in 1997 from where I was marshaling, seeming to be riding on smooth pavement through sections I rattle and bounce over, sure it was a shame when he left mountain biking, I am after all a mountain biker at heart, but there were as we all know bigger achievements in his future, there is nothing in the mtb world that could possibly excite a nation the way he has the last few years.

For the entire tour this year Cadel and his team seemed to be well organised, know what they were doing and went about everything the right way. Leopard also dealt with the race well, thus as Andy Schleck has said it definitely seems the best rider did indeed win this year. The final time trial was an incredible hour of viewing, seeing Cadel so focused and confidant at the start and then he almost won the stage and blasted away everyone else. Of course seeing him get air on a time trial bike was pretty cool too.

I really hope this helps move Australian's recognition of bikes and cycling forward, the reception for Cadel in Melbourne on Friday was awesome, with St Kilda rd lined 5 deep on each side all the way along and then Federation square packed so full along with all of us watching who did not make it down. The media coverage across the board has been positive and pretty good. Now we can all hope for a repeat performance next year. Rock on Cadel.

Steven Hanley: [mtb/gear] More Mont awesomeness

Tue, 2015-11-10 16:25

New and Old Zing Vests (fullsize) By far one of my favourite pieces of clothing is my Mont Zing Vest, I bought my first back in March 2006, then it was shredded in a crash in April 2008, I had however bought a second vest at that point, in yellow. Most cycling vests have mesh backs, I dislike these as I wear my vest to paddle and a waterproof back is great paddling. Also for warmth year round in all manner of activities (running, rogaines, etc) the lack of mesh is a bonus I think.

Due to the fact I use the vest so often through the colder months it is often damp or wet when I want to use it (soaking from paddling to ride home in winter, or from wearing for a run to get home), thus I was keen to get a few more. Mont are a great company and Dave has been very nice to me in the past. They had run out of Zing vests in my size and I had been asking Dave when I would be able to buy two more for a while. He had some ready for the yearly sale that is on next week and told me to come visit. However he had only had yellow vests made, which I was keen to buy another one of, I was however hoping for some colour variety. Dave then offered to make a few out of whatever colour Hydronaute ultra they had in the factory. Sure it is not pink but I am definitely partial to purple too so I was excited to be able to get two brand new vests in purple.

Did the Ainslie run up at lunch today, was letting my HR creep above 180 and then remembered I should not be going too hard so backed it off to 176 or so for the rest of the run up. Still did 15:16 which I thought was alright.

Steven Hanley: [comp] Obscurity, P=NP etc, Hash Visualisation

Tue, 2015-11-10 16:25
Three things I saw online today I feel like mentioning, first linked from Schneier's blog was an article about how lock making companies are still very much in the security through obscurity world and how lock geeks getting together online and at (computer) security conferences are breaking their obscure secrets open. An interesting read.

It is interesting to see some companies such as Kryptonite eventually reacted, others seem intent on denying public information, or trying to shut down people who know about it. In computing it is a well known fact (although still ignored by too many people/companies) that security through obscurity will not work, public design and analysis by experts in the field however does work and should be used for things that need to be secure. Although one aspect that comes to mind here is that in the case of locks you may not want to make them impossible as other attack vectors are then used. As the article mentions crooks seem to prefer using a hammer (or maybe explosives) over opening the locks through lock exploits. There were some discussions about this in the car that were I think linked to by Schneier a few years back.

Next was an interesting wikipedia page linked to by kottke, a list of unsolved problems from a number of different field, those listed in Computing are familiar, however looking through the collected information on those in other fields is pretty fascinating. Mmmmmm wikipedia goodness.

Catching up on some LWN reading and I see the mention of a new OpenSSH version approaching, in the list of new features is "Experimental SSH fingerprint visualisation" with a paper (pdf) linked. So I download and had a read of the paper, largely to see what sort of images they generate. It is good to see some work on what is one of the biggest security weaknesses out there, the humans using secure systems.

Steven Hanley: [mtb] Caffeine and glycogen storage, maybe the roadies have it right

Tue, 2015-11-10 16:25
There is an article in the Canberra Times today referring to a Melbourne study on some endurance cyclists measuring glycogen storage in muscles when caffeine is consumed immediately after a ride until exhaustion session.

The professor in charge of the study at RMIT is John Hawley, a google search turned up the article in question. It is quite a common practice among cyclists to head to a coffee shop after a ride, though most of us do not consume 6 cups of coffee and a loaf of bread, we do consume some food and coffee at these gatherings fairly often.

Good to see we can even claim the post ride coffee is part of our important training schedule and recovery plan.

Steven Hanley: [mtb/events] Out of Range at the 2007 Geoquest Adventure Race

Tue, 2015-11-10 16:25
As mentioned a few times the report from Geoquest 2007 has been a while in coming, pretty much all involved in the team have now seen it and have no problems with it. In it are 99 photos, 5 videos and a fair chunk of text. We had fun and I am hoping to be back next year for more of it. Thanks to Bruce, Danealle, Craig and Brendan for racing and big thanks to Jane, Zoe and Jaymz for supporting. Also Gran and Jude were fun to have around the race.

Anyway for anyone who wants to have a look here is our report from the team Out of Range at the 2007 Geoquest Adventure Race. Enjoy.

Steven Hanley: [various] Through the pearly gates in a 200 mph fireball

Tue, 2015-11-10 16:25
Anyone who recognises the quote probably has already guessed I watched Top Gear last night (the expression about going through the pearly gates in a fireball seems to be a favourite of the presenters). I agree with so many other viewers that this is a funny show. Heck I tend to have an almost negative interest in cars and yet this show has me laughing along throughout most episodes.

I remember watching the old series with Clarkson in it when I was living in the UK in 1993, though at the time I paid some small interest in cars (such as watching Ayrton Senna in F1 races) I do not recall Top Gear being so amusing. I suspect they really ramped up the humour of it when they changed the format and started the new series in 2002. My amusement at the pearly gates expression has me trying to think of a few expressions for how various people may want to go, somewhat macabre maybe but I am trying to think of it in a similar manner to my Fairy Tales in the key of Klingon post. Alas nothing comes to mind yet.