Planet Linux Australia

Syndicate content
Planet Linux Australia - http://planet.linux.org.au
Updated: 14 min 35 sec ago

Tim Serong: Salt and Pepper Squid with Fresh Greens

Sun, 2014-11-16 18:27

A few days ago I told Andrew Wafaa I’d write up some notes for him and publish them here. I became hungry contemplating this work, so decided cooking was the first order of business:

It turned out reasonably well for a first attempt. Could’ve been crispier, and it was quite salty, but the pepper and chilli definitely worked (I’m pretty sure the chilli was dried bhut jolokia I harvested last summer). But this isn’t a post about food, it’s about some software I’ve packaged for managing Ceph clusters on openSUSE and SUSE Linux Enterprise Server.

Specifically, this post is about Calamari, which was originally delivered as a proprietary dashboard as part of Inktank Ceph Enterprise, but has since been open sourced. It’s a Django app, split into a backend REST API and a frontend GUI implemented in terms of that backend. The upstream build process uses Vagrant, and is fine for development environments, but (TL;DR) doesn’t work for building more generic distro packages inside OBS. So I’ve got a separate branch that unpicks the build a little bit, makes sure Calamari is installed to FHS paths instead of /opt/calamari, and relies on regular packages for all its dependencies rather than packing everything into a Python virtualenv. I posted some more details about this to the Calamari mailing list.

Getting Calamari running on openSUSE is pretty straightforward, assuming you’ve already got a Ceph cluster configured. In addition to your Ceph nodes you will need one more host (which can be a VM, if you like), on which Calamari will be installed. Let’s call that the admin node.

First, on every node (i.e. all Ceph nodes and your admin node), add the systemsmanagement:calamari repo (replace openSUSE_13.2 to match your actual distro):

# zypper ar -f http://download.opensuse.org/repositories/systemsmanagement:/calamari/openSUSE_13.2/systemsmanagement:calamari.repo

Next, on your admin node, install and initialize Calamari. The calamari-ctl command will prompt you to create an administrative user, which you will use later to log in to Calamari.

# zypper in calamari-clients # calamari-ctl initialize

Third, on each of your Ceph nodes, install, configure and start salt-minion (replace CALAMARI-SERVER with the hostname/FQDN of your admin node):

# zypper in salt-minion # echo "master: CALAMARI-SERVER" > /etc/salt/minion.d/calamari.conf # systemctl enable salt-minion # systemctl start salt-minion

Now log in to Calamari in your web browser (go to http://CALAMARI-SERVER/). Calamari will tell you your Ceph hosts are requesting they be managed by Calamari. Click the “Add” button to allow this.

Once that’s complete, click the “Dashboard” link at the top to view the cluster status. You should see something like this:

And you’re done. Go explore. You might like to put some load on your cluster and see what the performance graphs do.

Concerning ceph-deploy

The instructions above have you manually installing and configuring salt-minion on each node. This isn’t too much of a pain, but is even easier with ceph-deploy which lets you do the whole lot with one command:

ceph-deploy calamari connect --master <calamari-fqdn> <node1> [<node2> ...]

Unfortunately, at the time of writing, we don’t have a version of ceph-deploy on OBS which supports the calamari connect command on openSUSE or SLES. I do have a SUSE-specific patch for ceph-deploy to fix this (feel free to use this if you like), but rather than tacking that onto our build of ceph-deploy I’d rather push something more sensible upstream, given the patch as written would break support for other distros.

Distros systemsmanagement:calamari Builds Against

The systemsmanagement:calamari project presently builds everything for openSUSE 13.1, 13.2, Tumbleweed and Factory. You should be able to use the packages supplied to run a Calamari server on any of these distros.

Additionally, I’m building salt (which is how the Ceph nodes talk to Calamari) and diamond (the metrics collector) for SLE 11 SP3 and SLE 12. This means you should be able to use these packages to connect Calamari running on openSUSE to a Ceph cluster running on SLES, should you so choose. If you try that and hit any missing Python dependencies, you’ll need to get these from devel:languages:python.

Disconnecting a Ceph Cluster from Calamari

To completely disconnect a Ceph cluster from Calamari, first, on each Ceph node, stop salt and diamond:

# systemctl disable salt-minion # systemctl stop salt-minion # systemctl disable diamond # systemctl stop diamond

Then, make the Calamari server forget the salt keys, ceph nodes and ceph cluster. You need to use the backend REST API for this. Visit each of /api/v2/key, /api/v2/server and /api/v2/cluster in your browser. Look at the list of resources, and for each item to be deleted, construct the URL for that and click “Delete”. John Spray also mentioned this on the mailing list, and helpfully included a couple of screenshots.

Multiple Cluster Kinks

When doing development or testing, you might find yourself destroying and recreating clusters on the same set of Ceph nodes. If you keep your existing Calamari instance running through this, it’ll still remember the old cluster, but will also be aware of the new cluster. You may then see errors about the cluster state being stale. This is because the Calamari backend supports multiple clusters, but the frontend doesn’t (this is planned for version 1.3), and the old cluster obviously isn’t providing updates any more, as it no longer exists. To cope with this, on the Calamari server, run:

# calamari-ctl clear --yes-i-am-sure # calamari-ctl initialize

This will make Calamari forget all the old clusters and hosts it knows about, but will not clear out the salt minion keys from the salt master. This is fine if you’re reusing the same nodes for your new cluster.

Sessions to Attend at SUSECon

SUSECon starts tomorrow (or the day after, depending on what timezone you’re in). It would be the height of negligence for me to not mention the Ceph related sessions several of my esteemed colleagues are running there:

  • FUT7537 – SUSE Storage – Software Defined Storage Introduction and Roadmap: Getting your tentacles around data growth
  • HO8025 – SUSE Storage / Ceph hands-on session
  • TUT8103 – SUSE Storage: Sizing and Performance
  • TUT6117 – Quick-and-Easy Deployment of a Ceph Storage Cluster with SLES – With a look at SUSE Studio, Manager and Build Service
  • OFOR7540 – Software Defined Storage / Ceph Round Table
  • FUT8701 – The Big Picture: How the SUSE Management, Cloud and Storage Products Empower Your Linux Infrastructure
  • CAS7994 – Ceph distributed storage for the cloud, an update of enterprise use-cases at BMW

Update: for those who were hoping for an actual food recipe, please see this discussion.

Binh Nguyen: Writing eBooks For Profit

Sun, 2014-11-16 03:15
Over the years it's been clear that I've had a propensity for writing. What hasn't been so clear was how to monetise this. Recent research has indicated that if you're a writer it isn't as difficult as you think. If you work with standard word processors and office suites then it becomes clear that it's possible basically to just type things up, export to PDF, and then publish this.

- a good example of this are the 'Building a Cloud Computing Service', 'Convergence Effect', and 'Cloud and Internet Security' (has been cleared by Australian Intelligence Services for sensitive material so it's not a problem if you're curious) reports which are now available via Amazon and Google Play Book stores for 5 USD each (pretty decent content and research for the price to be honest. Will be curious to see how this experiment goes...)

https://play.google.com/store/books/author?id=Binh+Nguyen

http://www.amazon.com/mn/search/?_encoding=UTF8&camp=1789&creative=390957&field-author=Binh%20Nguyen&linkCode=ur2&search-alias=digital-text&sort=relevancerank&tag=bnsb-20&linkId=3BWQJUK2RCDNUGFY

CLOUD AND INTERNET SECURITY: SECURITY MATTERS [Kindle Edition]http://www.amazon.com/gp/product/B00PKU2B4I/ref=as_li_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=B00PKU2B4I&linkCode=as2&tag=bnsb-20&linkId=CELT4LN4FINTOIEE

Building a Cloud Computing Service: An Introduction to the Cloud [Kindle Edition]http://www.amazon.com/gp/product/B00PKOZ8GM/ref=as_li_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=B00PKOZ8GM&linkCode=as2&tag=bnsb-20&linkId=LDVSS3FSJBYKXHWX

CONVERGENCE EFFECT: ACCUMULATING CLOUD SYSTEMS [Kindle Edition]

http://www.amazon.com/gp/product/B00PKVPJU0/ref=as_li_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=B00PKVPJU0&linkCode=as2&tag=bnsb-20&linkId=IVWXNS67IHG3QGQS



For those who are curious here are some interesting notes:- there are some plugins and standalone applications which will allow for this but at the end of the day you need to be able to run your book through the automated checkers to be able to get anything actually posted on to the online store

http://extensions.libreoffice.org/extension-center/elaixhttp://extensions.libreoffice.org/extension-center/writer2xhtmlhttp://soft.alkinea.net/https://wiki.documentfoundation.org/EPUB/Converting_ODF_to_EPUBhttp://calibre-ebook.com/http://www.ilearnthings.com/2013/10/17/episode-12-create-epub-documents-in-libreoffice/http://www.linux-magazine.com/Online/Blogs/Off-the-Beat-Bruce-Byfield-s-Blog/From-LibreOffice-to-ePubhttp://connect.lulu.com/t5/eBooks/bd-p/ebook_formatting- another option could be paying someone to manually convert your chosen file. The problem is that you never know the quality of the work that you're going to get so I suggest going on your own- most if not all stores will take a cut of what you sellhttps://kdp.amazon.com/

https://www.apple.com/itunes/https://play.google.com/bookshttps://play.google.com/books/publish/

- though there are other options out there if you want to sell in a different wayhttps://selz.com/https://www.lulu.com/https://www.createspace.com/http://www.cnet.com/how-to/how-to-self-publish-an-ebook/

http://www.jeffbullas.com/2013/10/08/should-you-sell-your-ebook-on-amazon-or-your-blog/- some sites will ask for ISBN details while others will supply them for you for free

http://www.isbn.org/about_ISBN_standardhttp://www.isbn.org/http://www.nla.gov.au/the-australian-isbn-agency- it's useful to know some details about how how to categorise the book in question so that you can target the right audience in the book storehttp://www.loc.gov/standards/sourcelist/subject-category.htmlhttp://www.bic.org.uk/7/BIC-Standard-Subject-Categories/

http://editeur.dyndns.org/bic_categories/344- use proper file converters if if you can not extract properlyhttp://www.adobe.com/products/acrobat/pdf-to-word-doc-converter.htmlhttp://www.adobe.com/products/acrobat/word-to-pdf-converter.htmlhttps://www.acrobat.com/en_us/free-trial-download.html?trackingid=KLARA- depending on your status you may need to sign up to have a Tax File Number in the United States. There are often taxation agreements with more developed countries thoughhttp://www.irs.gov/Individuals/International-Taxpayers/Taxpayer-Identification-Numbers-%28TIN%29

http://www.irs.gov/Individuals/2013-ITIN-Updated-Procedures-Frequently-Asked-Questionshttps://www.apple.com/uk/itunes/working-itunes/sell-content/music-faq.htmlhttps://sa.www4.irs.gov/modiein/individual/please-tell.jsphttp://www.irs.gov/uac/Where-to-File-Your-Taxes--(for-Form-SS-4)https://www.apple.com/uk/itunes/working-itunes/sell-content/books/book-faq.htmlhttp://www.irs.gov/Businesses/Small-Businesses-&-Self-Employed/How-to-Apply-for-an-EINhttp://www.irs.gov/Businesses/Small-Businesses-&-Self-Employed- many book stores will require you to use specific file formats or applicationshttps://itunesconnect.apple.com/docs/UsingiTunesProducerPaidBooks.pdfhttp://en.wikipedia.org/wiki/IBookshttps://www.apple.com/itunes/working-itunes/sell-content/books/book-faq.htmlhttps://www.apple.com/itunes/working-itunes/sell-content/books/https://www.apple.com/itunes/working-itunes/sell-content/- note that there are are many options/programs out there that will let you preview, manage, and convert your eBookshttp://fbreader.org/http://calibre-ebook.com/http://www.cultofmac.com/254741/how-to-add-epub-books-to-your-ipad-without-itunes-ios-tips/

David Rowe: OpenRadio Part 2 – Prototype Works!

Fri, 2014-11-14 14:30

Since the first post on the OpenRadio project Mark has been moving ahead and leaps and bounds. In just a few late nights work he has assembled and tested the radio, managed to receive off air signals, and even tested the PSK31 transmitter! Fine business Mark.

Mark writes:

Hooked it up to a real antenna tonight:

That’s me decoding actual 20m PSK31 signals!

Signal path is:

Antenna —RF—-> OpenRadio SDR —-IQ–>|Laptop|—-IQ—> Spectravue

(IQ Demod) —SSB—> fldigi

Mark managed to build the radio in 1-2 hours, including taking plenty of photos to document assembly and aid others. Now not everyone will have Mark’s radio assembly skills. However even allowing for a learning curve and a few coffee breaks we are on track for a one-day (say 6 hour) mini-conf assembly time.

A working prototype verifies the hardware design, so we are now getting ready to re-spin the PCB and start putting the kits together.

In other news Edwin from Dragino has added the OpenRadio kit to his store. We estimate the kits will be available for shipping in December. Kim, Mark and I, are still deciding if we will bring a bunch of kits to LCA, or have delegates pre-order them from Dragino. More on that shortly.

Links

OpenRadio Wiki

linux.conf.au News: LCA 2015 Penguin Dinner - The Theme

Fri, 2014-11-14 12:27

We're not sure if you've noticed but we all tend to have more than just a trifling interest in things technological and futuristic and, well... How do we put this? Some people have even felt moved to describe us as geeks. Can you believe it? Us?!

We thought for the 2015 Penguin Dinner maybe we should put that myth to rest, finally, and show that we do, indeed, conform to the norm. We're going to do something more traditional, more what you'd expect of a respectable, conservative "black tie dinner".

So it is with some seriousness and gravity, Lords, Ladies and Gentleman, that we announce, for the first time the LCA 2015 Penguin Dinner will have a theme:

Steampunk!

We would love it if you would dress for the occasion. It adds a certain air to the evening, however it is not mandatory (we wouldn't do that to you!). Costumes are suitable, gadgets will be welcome, the more quirky and fantastical the better! We know you're among the cleverest, most inventive and imaginative people to be found (not that we're biased!) so we can't wait to see you as futuristic and anachronistic characters and marvel at your latest inventions and innovations!





Stewart Smith: C bitfields considered harmful

Fri, 2014-11-14 11:26

In C (and C++) you can specify that a variable should take a specific number of bits of storage by doing “uint32_t foo:4;” rather than just “uint32_t foo”. In this example, the former uses 4 bits while the latter uses 32bits. This can be useful to pack many bit fields together.

Or, that’s what they’d like you to think.

In reality, the C spec allows the compiler to do just about anything it wants with these bitfields – which usually means it’s something you didn’t expect.

For a start, in a struct -e.g. “struct foo { uint32_t foo:4; uint32_t blah; uint32_t blergh:20; }” the compiler could go and combine foo and blergh into a single uint32_t and place it somewhere… or it could not. In this case, sizeof(struct foo) isn’t defined and may vary based on compiler, platform, compiler version, phases of the moon or if you’ve washed your hands recently.

Where this can get interesting is in network protocols (OMG DO NOT DO IT), APIs (OMG DO NOT DO IT), protecting different parts of a struct with different mutexes (EEP, don’t do it!) and performance.

I recently filed MySQL bug 74831 which relates to InnoDB performance on POWER8. InnoDB uses C bitfields which are themselves bitfields (urgh) for things like “flag to say if this table is compressed”. At various parts of the code, this flag is checked.

When you apply this simple patch:

--- mysql-5.7.5-m15.orig/storage/innobase/include/dict0mem.h +++ mysql-5.7.5-m15/storage/innobase/include/dict0mem.h @@ -1081,7 +1081,7 @@ struct dict_table_t { Use DICT_TF_GET_COMPACT(), DICT_TF_GET_ZIP_SSIZE(), DICT_TF_HAS_ATOMIC_BLOBS() and DICT_TF_HAS_DATA_DIR() to parse this flag. */ - unsigned flags:DICT_TF_BITS; + unsigned flags;

I get 10,000 key lookups/sec more than without it!

Why is this? If you go and read the bug, you’ll see that the amount of CPU time spent on the instruction checking the bit flag is actually about the same… and this puzzled me for a while. That is, until Anton reminded me that the PMU can be approximate and perhaps I should look at the loads.

Sure enough, the major difference is that with the bitfield in place (i.e. MySQL 5.7.5 as it stands today), there is a ld instruction doing the load – which is a 64bit load. In my patched version, it’s a lwx instruction – which is a 32bit load.

So, basically, we were loading 8 bytes instead of 4 every time we were checking if it was a compressed table.

So, along with yesterday’s lesson of never, ever, ever use volatile, today’s lesson is never, ever, ever use bitfields.

Andrew Pollock: [life] Day 288: A day of vomit, canceled play dates and bus excursions

Thu, 2014-11-13 22:25

I'd made plans with Kelley, the mum of Chloe, who is also starting Prep next year, to go on a day trip over to the Redcliffe lagoon for a swim and a picnic. Despite it raining overnight, and it being overcast in the morning, I was confident we could still make a good day of it.

Then Zoe threw up rather spectacularly before breakfast. At least she had the decency to do it in the kitchen on the tiles, so it was easy to clean up.

At that stage, I wasn't sure if it was a gastro bug or what, so I canceled the play date to be on the safe side. Zoe kept breakfast down, and after doing a bit of craft (we made a really awesome little castle out of a tissue box and some toilet rolls), I decided it was okay to venture out.

Zoe's Kindergarten is doing some sort of Christmas production, and she was asked to provide a plain green t-shirt for it. Big W was apparently selling such t-shirts for the bargain price of $3, so I thought we might as well head out there. Zoe was keen to take a bus. We've never tried a multiple bus excursion before, but I figured it would be a good way to pass the day. There was a bus due in 10 minutes of hatching this plan, so we hastily headed out the door.

We took the usual city-bound 230 bus from Hawthorne, and had to change for a Carindale-bound bus at Woolloongabba, which struck me as a really inefficient way to travel, but it was what Google Maps said to do, so we did it. On the bus to Carindale, my friend Steve called me up because he had an hour to kill, and we arranged to meet up at Carindale for a coffee.

By the time we got there it was pretty much time for an early lunch, so we grabbed some lunch in the Glasshouse precinct of Westfield and Zoe had a play in the play area. I also tried out the selfie stick I'd picked up a few weeks ago. Zoe thought it was pretty cool.

After Steve departed, we bought the t-shirt, and headed back to the buses. It was nice to just wander around Carindale without a particular agenda for a change.

Having been the first time we'd traveled to Carindale by bus, we got on the 204 bus going in the wrong direction, and I didn't realise until we reached the end of the line. We had a brief wait for the next bus to leave going back the right way. I had a bit of a chat with the bus driver, who had a granddaughter about Zoe's age. I learned that 6 bus drivers a week get bashed, which seems rather dreadful. I also learned that they've been told it causes less fuss to just let passengers travel with a Go card with insufficient credit, so I'm not sure why I'm bothering to auto top up mine.

With the unplanned extra delay, we got home running late for swim class, so we drove to swim class, and Zoe had her swimming lesson.

After a brief side trip to the Hawthorne Garage to grab some stuff for dinner, we headed home, and Zoe watched a bit of TV while I put dinner on.

Bedtime was nice and uneventful.

OpenStack miniconf: linux.conf.au 2015 OpenStack miniconf call for presentations

Thu, 2014-11-13 18:26
linux.conf.au will be hosting the fourth OpenStack mini-conference to run in the Australia / New Zealand region. Excitingly, this is the first OpenStack mini-conference to be run in New Zealand. The idea for next years linux.conf.au miniconf is to “pivot” from previous miniconfs and present something which is clearly targetted at deployers and system admins. This is based on feedback that conference attendees believed the miniconf to be a developer meetup, and therefore weren’t attending. The plan is for the miniconf to follow a logical flow of things a deployer would be thinking about as they install OpenStack in their existing environment. The miniconf will be recorded, so hopefully this will form a useful resource for others. The mini-conference is therefore calling for proposals for content. Speakers at the mini-conference must be registered for linux.conf.au 2015 as delegates, or discuss their needs with the mini-conference organizers if that isn’t possible. Proposed miniconf flow:
  • What are the components of OpenStack and what order should I deploy them in? What problems do each component solve?
    • Layer 1: keystone, glance, nova, neutron
    • Layer 2: cinder, ironic
    • Layer 3: horizon, ceilometer
    • Swift (its not clear what layer this will end up in, we will handle it separately)
  • Configuration management options:
    • puppet
    • chef
    • ansible
    • juju
  • Keystone:
    • integrating auth with your existing auth systems (LDAP, Active Directory)
  • Nova:
    • hypervisor options
    • instance storage options
  • Neutron:
    • network deployment options
    • gotchas
  • Cinder
  • Ironic
  • Horizon
    • Theming to match your corporate look
    • Adding panels
  • Ceilometer
  • Swift
  • Operational matters:
    • monitoring
    • logging
    • debugging
    • where to get help
For your proposed talk, please specify where in this flow it would fit.

Some important details:

  • linux.conf.au runs from 12 to 16 January 2015 in Auckland, New Zealand
  • the mini-conference will be on Tuesday the 13th of January
  • proposals are due to the mini-conference organiser no later than 4 December 2014
  • there are two types of talks — full length (45 minutes) and half length (20 minutes), because of the intention to follow a logical flow some negotiation is expected around the content and duration of talks proposed
CFP submissions are made by completing this online form: https://docs.google.com/forms/d/1pNXwmw8iwNFRTp_yZq0yVvyA6_KRyrp-hU9PITJCqNs/viewform If you have questions about this call for presentations, please contact Michael Still at openstack-lca2015@lists.stillhq.com for more details.

Stewart Smith: volatile considered harmful

Thu, 2014-11-13 11:26

While playing with MySQL 5.7.5 on POWER8, I came across a rather interesting bug (74775 - and this is not the only one… I think I have a decent amount of auditing and patching to do now) which made me want to write a bit on memory barriers and the volatile keyword.

Memory barriers are hard.

Like, super hard. It’s the kind of thing that makes you curse hardware designers, probably because they’re not magically solving all your problems for you. Basically, as you get more CPU cores and each of them have caches, it gets more expensive to keep everything in sync. It’s quite obvious that with *ahem* an eventually consistent model, you could save a bunch of time and effort at the expense of shifting some complexity into software.

Those in the MySQL world should recognize this – we’ve been dealing with asynchronous replication for well over a decade as a good way to scale.

On some CPU architectures (POWER for example) not all loads are created equal. When you load a value from memory, it will be consistent with your thread of execution. That is, with any stores that you have done in this thread of execution. If another thread updates that memory location you may not see that update even if your load occurs after that thread updates that memory location. Think eventually consistent.

If you want up to date reads (and not clobber writes), then you get to do memory barriers! (a topic for elsewhere – the PowerISA document has good explanations of what we have on POWER though, and how load with reserve works).

What the volatile keyword does is generate load and store instructions. It is useful when talking to hardware, as the load and store instructions are actually doing something there that the compiler doesn’t know about and thus shouldn’t optimize away.

The volatile keyword does not add any memory barriers. This is important to realize – volatile just makes loads and stores happen for your thread, not in relation to any other threads of execution. Thus, you cannot use volatile as a thread synchronization mechanism at all. It is completely and totally wrong.

Basically, if you have a volatile variable and you do stores to it in one thread and loads in another, after the store happens, it could be quite a long time before the thread doing the loads sees it! For some applications this may be okay (although I can’t really think of any beyond very very inaccurate status variables)… but if it matters at all for application correctness, volatile is the wrong thing to use.

Further reading:

linux.conf.au News: Speaker Feature: Leslie Hawthorn, Richard Jones

Thu, 2014-11-13 07:27
Leslie Hawthorn Checking Your Privilege: A How-To for Hard Things

2:15pm Wednesday 14th January 2015

An internationally known community manager, speaker and author, Leslie has spent the past decade creating, cultivating and enabling open source communities. She created the world’s first initiative to involve pre-university students in open source software development, launched Google’s #2 Developer Blog, received an O’Reilly Open Source Award in 2010 and gave a few great talks on many things open source.

In August 2013, she joined Elasticsearch as Community Manager, where she leads Developer Relations. She works from Elasticsearch’s EU HQ in Amsterdam, The Netherlands – when not out and about gathering user praise and pain points.

For more information on Leslie's presentation, see here.



Richard Jones The future of Python packaging

1:20pm Thursday 15th January 2015

Richard has been a Python programmer and part of the Python community for a very long time now. He created and administers pypi.python.org, though he has help from infrastructure elves these days.

For more information on Richard and his presentation, see here. You can follow him as @r1chardj0n3s and don’t forget to mention #LCA2015.

Andrew Pollock: [life] Day 287: Kindergarten, another Prep day and a big dinner

Wed, 2014-11-12 22:25

I was up and about very early again this morning. I started the day off with my chiropractic adjustment and got stuck into the last part of the current unit of my real estate licence course.

The cleaners were about 15 minutes late arriving, and I had to race out the door as soon as they arrived to have a one-on-one with my group leader.

That went okay, and I got home and did some more of my coursework before heading out for my massage. I was feeling really sore after my run and my yoga class, so I really needed it this week.

After that, I had enough time to grab some lunch and race over to pick up Zoe from Kindergarten early to take her to Prep for her third introductory session. We introduced ourselves to the Mum of a little boy who was also heading to school and recognised Zoe in her Prep t-shirt.

The session went much better than the first one, probably because we were less rushed getting there on time and Zoe was well and truly with the program by now.

After school, we went back to the uniform shop to return one dress too many, and then ducked over to the supermarket to get some stuff for dinner. I'd arranged for Megan and her sister and Dad to come for dinner, as Laura was out of town. At school, I ran into Eva and Layla's Mum, Tanya, and Justin was out of town for a funeral, so I invited them over for dinner as well.

Five kids in a small apartment was quite the zoo, but everyone had a good dinner, play and bath (I need a few more towels to deal with that many bathing guests). Zoe was pretty tired, and with all the excitement I didn't get her down to bed until a little bit late.

Unfortunately, I didn't quite get the unit of study completed that I was working on today. I'll have to try and finish it off over the next couple of days.

Stewart Smith: Preliminary MySQL Cluster benchmark results on POWER8

Wed, 2014-11-12 16:27

Yesterday, I got the basics going for MySQL Cluster on POWER. Today, I finished up a couple more patches to improve performance and ran some benchmarks.

This is on a 3.7Ghz POWER8 machine with non-balanced memory (only 2 of the 4 NUMA nodes have memory, so we have less total memory bandwidth than we could have, plus I’m going to bind ndbmtd to the CPUs in these NUMA nodes)

With a setup of a single replica and two data nodes on the one machine (each bound to a specific NUMA node), running the flexAsync benchmark on MySQL Cluster 7.3.7, I could get around:

  • 3.2 million reads/sec
  • 2.6 million deletes/sec
  • 2.4 million updates/sec
  • 2.4 million inserts/sec.

So, that’s at least in the right ballpark for a first go.

(I’m running this on a big endian host kernel, some random kernel I booted on the box and built with gcc 4.8 with whatever build options the MySQL Cluster cmake foo chooses by default)

linux.conf.au News: Speaker Feature: Deb Nicholson, Julian Simpson

Wed, 2014-11-12 07:28
Deb Nicholson Software Patents: Trolls and Other Bullies

11:35am Wednesday 14th January 2015

Deb wants to make the world a better place with technology and social justice for all. After many years of local political organizing, she started handling outreach for the Free Software Foundation and became an enthusiastic free software activist. She likes talking to developers about software patents, to project maintainers about leadership and to activists about free software. She is currently the Community Outreach Director at the Open Invention Network and the Community Manager at GNU MediaGoblin. She also serves on the board at Open Hatch, a.k.a. Free Software's Welcoming Committee. She lives in Cambridge, Massachusetts surrounded by a community of food nerds and noisy musicians.

Deb won the O’Reilly Open Source Award, one of the most recognized awards in the FLOSS world, for her work on GNU MediaGoblin and OpenHatch.

For more information on Deb and her presentation, see here. You can follow her as @mediagoblin and don’t forget to mention #LCA2015.



Julian Simpson Surrounded by Graphs - a short introduction to Graph Databases and Neo4j

1:20pm Friday 16th January 2015

Julian has had a long and interesting career in the very comfortable trenches of IT. From a very entry-level position in the industry, he's worked in Support, Unix Systems Administration, Build and Release Management, and Development. He has worked in New Zealand, the USA and the UK where he was foresighted enough to buy his EuroStar ticket to the first DevOpsDays conference in 2009.

Julian has worked at many companies as an employee and as a consultant. Most of his experience is in Telecommunications, Media, and Finance. For the last 3 years he's worked for Neo Technology, the people who make Neo4j. He lives in Auckland with his wife and family.

For more information on Julian and his presentation, see here. You can follow her as @builddoctor and don’t forget to mention #LCA2015.

Stewart Smith: MySQL Cluster on POWER8

Tue, 2014-11-11 23:26

So, I’ve written previously on MySQL on POWER, and today is a quick bit of news about MySQL Cluster on POWER – specifically MySQL Cluster 7.3.7.

I ran into three main issues in getting some flexAsync benchmark results. One of them was the fact that I wanted to do this in the middle of all the POWER8 machines I usually use moving buildings (hard to run benchmarks when computers are packed up in boxes on a truck).

The next issue was that ndbmtd (the multi-threaded data node) needs memory barriers for the magic message passing stuff between threads. So, that’s pretty easy (about an eight line patch).

The next issue was in the results from flexAsync, it turns out 32bit math is a bad idea with results from my POWER8 box.

My preliminary performance numbers are fairly promising (actually… what is the world record for a single machine and NDB these days? Single data node?). I think there’s a bit more low hanging fruit and a couple more things that are a bit more involved.

Bugs with patches:

  • Bug 74782 – compile fix (memory barriers for POWER)
  • Bug 74781 – flexAsync uses 32bit math, leading to incorrect summary on POWER8

Andrew Pollock: [life] Day 286: Kindergarten, startup stuff, uniform shopping

Tue, 2014-11-11 18:25

This morning was a bit cooler on account of it being overcast, so I managed to leap out of bed and go for a run. I realised afterwards that it's been quite a while since I've gone for one. I mustered 7 km this morning before blowing up, mostly due to a lack of willpower to keep slogging on in the heat. I was happy I lasted that long.

I made up for last week's lost progress on my real estate licence course and knocked over a unit and put it in the mail. I managed to get through the first two parts of another unit, and hopefully I can finish off the third part and get it into the mail tomorrow.

I also did some productive procrastinating and may actually have a successful backup of daedalus currently getting written out to virtual "tape". Who would have thought the TCP keepalive interval would be the cause of all the problems?

I biked to Kindergarten to pick Zoe up, and then we went to the post office. I thought we should get Zoe's school uniform shopping out of the way, so under protest, we headed to the uniform shop on the way home from the post office.

For some reason, Zoe hadn't been excited about the prospect of going uniform shopping. Every time I'd asked her if she wanted to do it, she'd declined. Once we got into the store and were trying on uniforms though, she wanted to wear one home on the bike. So that's the uniform shopping out of the way, we just have to get some shoes, which I'll leave until the last minute in case her feet grow.

After we got home, Zoe watched some TV and I had a crack at making a herb and garlic pull-apart that I've been wanting to try and make for a while. It's smelling delicious in the oven as I write.

I'm looking forward to closing out the day with a yoga class.

Glen Turner: USB Vendor ID for documentation

Tue, 2014-11-11 16:36

If you are writing documentation then you don't want to use an assigned magic number, like a real IP address or a real DNS name. That can readily lead to: misunderstandings; operational difficulties for the vendor's equipment if the number escapes from documentation into production; and difficulties for the author because of the risk of defamation and trademark infringement.

For these reasons standards associations commonly issue a range of their magic numbers for documentation purposes. For example, the IETF issued magic numbers for documentation in RFC2606 for DNS names, in RFC5737 for IPv4 addresses and in RFC3849 for IPv6 addresses.

I was writing some documentation for using udev, and rather than defame some vendor by suggesting that their product may need a workaround, I asked the USB Implementors' Forum if there is a USB Vendor ID for documentation purposes.

Sadly, there is not:

From: USB-IF Administration <redacted>

Subject: RE: Vendor-ID for use in documentation

Date: 11 November 2014 2:34:21 PM ACDT

To: Glen Turner <redacted>

Dear Glen,

Thank you for your message. Vendor IDs (VIDs) are owned by the vendor company and are assigned and maintained by the USB-IF only. We do not have a generic VID for documentation.

Regards, redacted

David Rowe: OpenRadio – a one day Software Defined Radio project

Tue, 2014-11-11 11:29

For the 2015 Linux Conference, I am working with Kim Hawtin and Mark Jessop on a 1 day Open Radio Mini-conference.

In this mini-conf a classroom of people will solder together their very own software defined radio (SDR) transceivers in just a few hours. It will be capable of receiving signals on the HF radio bands (3 to 30 MHz), and short range transmission of FSK/PSK data on the 13.5 and 27 MHz ISM bands (no license required).

The project is being documented on our OpenRadio Wiki. It’s completely open source and we have published the PCB CAD files, and the parts list with Digikey/Element14 catalogue part numbers. It’s based on the soft-rock radio designs.

We have put a lot of effort into making the radio easy to build. For example a minimum of (large footprint) surface mount parts, and a simple, fast to assemble design. We have intentionally included one or two inductors and transformers to wind to give people a taste of the complete radio assembly experience. With a little supervision, the project is quite suitable for radio/electronics beginners or school age children. It’s a “crystal set” for the 21st century.

Mark has done a great job designing the radio, and we have just received the prototype PCBs:

This week we will assemble and test the first prototypes, measure how long they take to build, and noting possible snags for inexperienced builders. Then our good friend Edwin from Dragino will prepare and ship kits for the mini-conf.

The resources we create for this project (wiki, CAD files, software, kits from Dragino) will remain available after LCA. So you, your radio club, hackerspace, or even school class will have access to an easy to build a Software Defined Radio (SDR).

linux.conf.au News: Speaker Feature: Fraser Tweedale, Peter Chubb

Tue, 2014-11-11 07:27
Fraser Tweedale FreeIPA: Open Source Identity Management

2:15pm Friday 16th January 2015

Fraser is a developer at Red Hat, where he works on the FreeIPA identity management solution and Dogtag Certificate System. He is passionate about security and privacy. In his spare time, Fraser writes a lot of Haskell and patiently awaits the strongly-typed functional programming revolution.

For more information on Fraser and his presentation, see here. You can follow his as @hackuador and don’t forget to mention #LCA2015.



Peter Chubb SD Cards and filesystems for Embedded Systems

2:15pm Friday 16th January 2015

Peter has been hacking on UNIX since 1979, and has never used Windows. He currently does system (kernel and low-level) programming in a Linux environment for NICTA.

Peter's research interests include operating system algorithms for scalability, including storage, scheduling, memory management, and locking. He is also interested in systems performance measurement and optimisation.

Related hobbies include music, photography and fine wines, these also occasionally lead to research.

For more information on Peter and his presentation, see here.

Andrew Pollock: [life] Day 285: Kindergarten and some tinkering

Mon, 2014-11-10 18:25

I've felt exceptionally flat today, despite having a good night's sleep. I'm blaming the heat, or treking around Saint Helena Island in the hot sun yesterday.

We biked to Kindergarten this morning for drop off, and I left the trailer there. I was feeling pretty flat just after biking there and back in the heat.

I'm trying to rig up something to bring to Kindergarten one day, which uses a Raspberry Pi and the MaKey MaKey. I spent the day fooling around with it, and not much else.

I biked back to Kindergarten to pick Zoe up. Zoe and Megan wanted to have a play date, and Jason had to run some errands, so we biked home, and he dropped Megan off.

The girls had a good time running amok, and I made a start on my first batch of fruit mince for mince pies of this Christmas season.

Jason came over to pick up Megan, and Sarah arrived not long afterwards.

Linux Users of Victoria (LUV) Announce: LUV Beginners November Meeting: Developing GUI applications

Mon, 2014-11-10 16:29
Start: Nov 15 2014 12:30 End: Nov 15 2014 16:30 Start: Nov 15 2014 12:30 End: Nov 15 2014 16:30 Location: 

RMIT Building 91, 110 Victoria Street, Carlton South

Link:  http://luv.asn.au/meetings/map

There are a few useful tools/IDEs available on Linux to develop GUI applications. They are all similar in features. In this talk Daniel Jitnah will briefly talk about how GUI applications work, and what are the toolkits available: GTK, QT and Tk as examples. He will also demonstrate how a very simple GUI application can be built. The IDEs used will be QTDesigner, Lazarus, Anjuta+Glade and Netbeans.

LUV would like to acknowledge Red Hat for their help in obtaining the Buzzard Lecture Theatre venue and VPAC for hosting.

Linux Users of Victoria Inc., is an incorporated association, registration number A0040056C.

November 15, 2014 - 12:30

read more

linux.conf.au News: Speaker Feature: Selena Deckelmann, Thomas Sprinkmeier

Mon, 2014-11-10 07:27
Selena Deckelmann What technical learners need

3:40pm Thursday 15th January 2015

Selena is a major contributor to PostgreSQL and a data architect at Mozilla. She is a director of the Python Software Foundation.

She's been involved with free and open source software since 1995 and began running conferences for PostgreSQL in 2007. In 2012, she founded PyLadiesPDX, a portland chapter of PyLadies. She founded Open Source Bridge, Postgres Open and speaks internationally about open source, databases and community. She also keeps chickens and gives a lot of technical talks.

For more information on Selena and her presentation, see here. You can follow her as @selenamarie and don’t forget to mention #LCA2015.



Thomas Sprinkmeier How to train your Minions

3:40pm Friday 16th January 2015

Thomas graduated from UniSA in 1992 as an Electronic Engineer where he was seduced by PC's early in first year.

He's been working as a Software Engineer ever since for Ebor Computing in a variety of projects, usually with heavy mathematical, signal processing and networking components, occasionally interfacing to the 'real world'. Most recently he has been working at making cars smarter an safer, on the assumption that this might be easier than upgrading drivers.

Thomas started embarassing his kids at school by taking over the class and teaching about things from pulleys to railguns, paper planes to robot programming, conducting playdough to tidal locks. Most recently he has been teaching on weekends about Raspberry Pi, Arduino and 3D printing.

For more information on Thomas and his presentation, see here.