Planet Linux Australia

Syndicate content
Planet Linux Australia - http://planet.linux.org.au
Updated: 9 min 39 sec ago

Rusty Russell: Wrapper for running perf on part of a program.

Fri, 2015-07-03 13:28

Linux’s perf competes with early git for title of least-friendly Linux tool.  Because it’s tied to kernel versions, and the interfaces changes fairly randomly, you can never figure out how to use the version you need to use (hint: always use -g).

But when it works, it’s very useful.  Recently I wanted to figure out where bitcoind was spending its time processing a block; because I’m a cool kid, I didn’t use gprof, I used perf.  The problem is that I only want information on that part of bitcoind.  To start with, I put a sleep(30) and a big printf in the source, but that got old fast.

Thus, I wrote “perfme.c“.  Compile it (requires some trivial CCAN headers) and link perfme-start and perfme-stop to the binary.  By default it runs/stops perf record on its parent, but an optional pid arg can be used for other things (eg. if your program is calling it via system(), the shell will be the parent).

Linux Users of Victoria (LUV) Announce: LUV Main July 2015 Meeting: Ansible / BTRFS / Educating People to become Linux Users

Thu, 2015-07-02 19:29
Start: Jul 7 2015 18:30 End: Jul 7 2015 20:30 Start: Jul 7 2015 18:30 End: Jul 7 2015 20:30 Location: 

200 Victoria St. Carlton VIC 3053

Link:  http://luv.asn.au/meetings/map

Speakers:

• Andrew Pam, An introduction to Ansible

• Russell Coker, BTRFS update

• Lev Lafayette, Educating People to become Linux Users: Some Key Insights from Adult Education

200 Victoria St. Carlton VIC 3053 (formerly the EPA building)

Before and/or after each meeting those who are interested are welcome to join other members for dinner. We are open to suggestions for a good place to eat near our venue. Maria's on Peel Street in North Melbourne is currently the most popular place to eat after meetings.

LUV would like to acknowledge Red Hat for their help in obtaining the venue and VPAC for hosting.

Linux Users of Victoria Inc. is an incorporated association, registration number A0040056C.

July 7, 2015 - 18:30

read more

Donna Benjamin: Certification: Necessary Evil?

Thu, 2015-07-02 14:27
Thursday, July 2, 2015 - 14:14

I wrote this as a comment in response to Dries' post about the Acquia certification program - I thought I'd share it here too. I've commented there before.

I've also been conflicted about certifications. I still am. And this is because I fully appreciate the pros and cons. The more I've followed the issue, the more conflicted I've become about it.



My current stand, is this. Certifications are a necessary evil. Let me say a little on why that is.

I know many in the Drupal community are not in favour of certification, mostly because it can't possibly adequately validate their experience.



It also feels like an insult to be expected to submit to external assessment after years of service contributing to the code-base, and to the broader landscape of documentation, training, and professional service delivery.



Those in the know, know how to evaluate a fellow Drupalist. We know what to look for, and more importantly where to look. We know how to decode the secret signs. We can mutter the right incantations. We can ask people smart questions that uncover their deeper knowledge, and reveal their relevant experience.



That's our massive head start. Or privilege. 



Drupal is now a mature platform for web and digital communications. The new challenge that comes with that maturity, is that non-Drupalists are using Drupal. And non specialists are tasked with ensuring sites are built by competent people. These people don't have time to learn what we know. The best way we can help them, is to support some form of certification.



But there's a flip side. We've all laughed at the learning curve cartoon about Drupal. Because it's true. It is hard. And many people don't know where to start. Whilst a certification isn't going to solve this completely, it will help to solve it, because it begins to codify the knowledge many of us take for granted.



Once that knowledge is codified, it can be studied. Formally in classes, or informally through self-directed exploration and discovery.



It's a starting point.



I empathise with the nay-sayers. I really do. I feel it too. But on balance, I think we have to do this. But even more, I hope we can embrace it with more enthusiasm.



I really wish the Drupal Association had the resources to run and champion the certification system, but the truth is, as Dries outlines above, it's a very time-consuming and expensive proposition to do this work.



So, Acquia - you have my deep, albeit somewhat reluctant, gratitude!



:-)



Thanks Dries - great post.



cheers,

Donna

(Drupal Association board member)

James Purser: Tell your MP you support Same Sex Marriage

Thu, 2015-07-02 10:30

If you support the right for two people to get married regardless of gender, then please respectfully and politely contact your local federal member and let them know.

Those who oppose this have already started up their very effective networks, and we will need to work very hard to counter it.

If you're not sure who your local MP or Senators are, I recommend you use http://www.openaustralia.org.au/ to find out. Just punch in your post code and it will let you know, as well as give you a run down of their voting history.

Do it, DO IT NOW!

This message brought to you by the realisation that I'm going to be rainbow haired soon.

Blog Catagories: Politicssame sex marriage

Michael Still: Hunting for GC1D1NB

Thu, 2015-07-02 09:28
I went for an after work walk to try and find GC1D1NB on Tuggeranong Hill yesterday. It wasn't a great success. I was in the right area but I just couldn't find it. Eventually I ran out of time and had to turn back. I am sure I'll have another attempt at this one soon.



   



Interactive map for this route.



Tags for this post: blog pictures 20150701-tuggeranong_hill photo canberra bushwalk

Related posts: Goodwin trig; Big Monks; Geocaching; Confessions of a middle aged orienteering marker; A quick walk through Curtin; Narrabundah trig and 16 geocaches



Comment

David Rowe: FreeDV Robustness Part 5 – FreeDV 700

Wed, 2015-07-01 15:30

We’ve just released FreeDV v0.98 GUI software, which includes the new FreeDV 700 mode. This new mode has poorer speech quality than FreeDV 1600 but is far more robust, close to SSB on low SNR fading HF channels. Mel Whitten and the test team have made contacts over 1000 km using just 1 Watt!

You can download the Windows version of FreeDV 0.98 here.

To build it you need the latest codec2-dev and fdmdv2-dev from SVN, follow the Quickstart 1 instructions in fdmdv-dev/README.txt. I’ve been cross compiling for Windows on my Ubuntu Linux machine which is a time saver for me. Thanks Richard Shaw for your help with the cmake build system.

Mel and the team have been testing the software for the past few weeks and we’ve removed most of the small UI bugs. Thanks guys! I’m working on some further improvements to the robustness which I will release in a few weeks. Once we are happy with the FreeDV 700 mode, it will be ported to the SM1000. If you have time, and gcc/embedded experience I’d love to have some help with this!

It’s sounds pretty bad at 700 bit/s but so does SSB at 0dB SNR. The new modem uses a pilot symbol assisted coherent PSK modem (FreeDV 1600 uses a differential PSK modem). The new modem also has diversity; the 7 x 75 symb/s QPSK carriers are copied to form a total of 14 half power carriers. Overall this gives us significantly lower operating point SNR than FreeDV 1600 for fading channels. However the bandwidth is a little wider (800 – 2400 Hz), lets see how that goes through real radios.

Simulations indicate it has readability 4/5 at 0dB SNR on CCIR poor (fast) fading channels. It also has a PAPR of 7dB so if your PA can handle it you can hammer out 5dB more power than FreeDV 1600 (be careful).

For those of you who are integrating FreeDV into your own applications the FreeDV API now contains the 700 bit/s mode and freedv_tx and freedv_rx have been updated to demo it. The API interface has changed, we now have variables for the number of modem and speech samples which change with the mode. The coherent PSK modem has the very strange sample rate of 7500 Hz which at this stage the user (that’s you) has to deal with (libresample is your friend).

The 700 bit/s codec (actually 650 bit/s plus 2 data bits/frame) band limits the input speech between 600 and 2200 Hz to reduce the amount of information we need to encode. This might be something we can tweak, however Mel and the team have shown we can communicate OK using this mode. Here are some samples at 1300 (the codec rate used in FreeDV 1600) and 700 bit/s with no errors for comparison.

Lots more to talk about. I’ll blog some more when I pause and take a breath.

Donna Benjamin: Comparing D7 and D8 outta the box

Wed, 2015-07-01 12:27
Wednesday, July 1, 2015 - 11:53

I did another video the other day. This time I've got a D7 and D8 install open side by side, and compare the process of adding an article.

Linux Australia News: Linux Australia council meeting minutes to be published on the planet

Wed, 2015-07-01 12:26
Wed, 2015-07-01 11:33

Last fortnight the Linux Australia council resolved to begin publishing their minutes to planet.linux.org.au.

While meeting minutes may seem boring, they in fact contain a lot of useful and interesting information about what the organisation and its various subcommittees are up to. As such we felt that this was useful information to publish wider and starting from now we'll be publishing them to the planet.

If you are interested in previous meetings and minute notes, you can find them at http://linux.org.au/news

David Rowe: New Charger for my EV

Wed, 2015-07-01 09:30

On Sunday morning I returned home and plugged in my trusty EV to feed it some electrons. Hmm, something is wrong. No lights on one of the chargers. Oh, and the charger circuit breaker in the car has popped. Always out for adventure, and being totally incompetent at anything above 5V and 1 Amp, I connected it directly to the mains. The shed lights started to waver ominously. Humming sounds like a Mary Shelley novel. And still no lights on the charger.

Oh Oh. Since disposing of my nasty carbon burner a few years ago I only have one car and it’s the EV. So I needed a way to get on the road quickly.

But luck was with me. I scoured my local EV association web site, and found a 2nd hand Zivan NG3 charger, that was configured for a 120V lead acid pack. I have a 36 cell Lithium pack that is around 120V when charged. Different batteries have different charging profiles, for example the way current tapers. However all I really need is a bulk current source, my external Battery Management System will shut down the charger when the cells are charged.

Using some residual charge I EVed down the road where I met Richard, a nice man, fellow engineer, and member of our local EV association. I arranged to buy his surplus NG3, took it home and fired it up. Away it went, fairly hosing electrons into my EV at 20A. The old charger was just 10A so this is a bonus – my charging time will be halved. I started popping breakers again, as I was sucking 2.4kW out of the AC. So I re-arranged a few AC wires, ripped out the older chargers, rewired the BMS module loop a little and away I went with the new charger.

Here is the lash up for the initial test. The new Zivan NG3 is the black box on the left, the dud charger the yellow box on the right. The NG3 replaces the 96V dud charger and two 12V chargers (all wired in series) that I needed to charge the entire pack. My current clamp meter (so useful!) is reading 17A.

Old chargers removed and looking a bit neater. I still need to secure the NG3 somehow. My BMS controller is the black box behind the NG3. It shuts down the AC power to the chargers when the batteries signal they are full.

Pretty red lights in the early morning. Each Lithium cell has a BMS module across it, that monitors the cell voltage, The red light means “just about full”. When the first cell hits 4.1V, it signals the BMS controller to shut down the charger. Richard pointed out that the BMS modules are shunt regulators, so will discharge each cell back down to about 3.6V, ensuring they are all at about the same state of charge.

This is the only reason I go to petrol stations. For air. There is so little servicing on EVs that I forget to check the air for a year, some tyres were a bit low.

The old charger lasted 7 years and was used almost every day (say 2000 times) so I can’t complain. The NG3 was $875 2nd hand. Since converting to the Lithium pack in 2009 I have replaced the electric motor armature (about $900) as I blew it up from overheating, 2 cells ($150 ea) as we over discharged them, a DC-DC converter ($200 ish) and now this charger. Also tyres and brakes last year, which are the only wearing mechanical parts left. In that time I’ve done 45,000 electric km.

Michael Still: Percival trig

Tue, 2015-06-30 21:29
I had a pretty bad day, so I knocked off early and went for a walk before going off to the meeting at a charity I help out with. The walk was to Percival trig, which I have to say was one of the more boring trigs I've been to. Some of the forest nearly was nice enough, but the trig itself is stranded out in boring grasslands. Meh.



   



Interactive map for this route.



Tags for this post: blog pictures 20150630-percival photo canberra bushwalk trig_point

Related posts: Goodwin trig; Big Monks; Narrabundah trig and 16 geocaches; Cooleman and Arawang Trigs; One Tree and Painter; A walk around Mount Stranger



Comment

Michael Still: A team walk around Red Hill

Tue, 2015-06-30 09:28
My team at work is trying to get a bit more active, so a contingent from the Canberra portion of the team went for a walk around Red Hill. I managed to sneak in a side trip to Davidson trig, but it was cheating because it was from the car park at the top of the hill. A nice walk, with some cool geocaches along the way.



 



Interactive map for this route.



Tags for this post: blog pictures 20150629-davidson photo canberra bushwalk trig_point

Related posts: Goodwin trig; Big Monks; Narrabundah trig and 16 geocaches; Cooleman and Arawang Trigs; One Tree and Painter; A walk around Mount Stranger



Comment

Binh Nguyen: The Value of Money - Part 4

Tue, 2015-06-30 01:25
- I previously remarked that since we use the concept of 'deterrence' so readily throughout the world we are in a de-facto state of 'Cold War' whose weapons are defense, intelligence, and economics. There's a lot of interesting information out there...

http://blogs.telegraph.co.uk/news/shashankjoshi/100224247/france-should-remember-its-own-history-before-complaining-too-much-about-american-espionage/

https://wikileaks.org/gifiles/docs/11/1172615_-ct-analysis-an-economic-security-role-for-european-spy.html

http://www.wikileaks-forum.com/nsa/332/r-james-woolsey-why-we-spy-on-our-allies-17-03-2000/24575/

http://www.abc.net.au/news/2013-11-08/australian-nsa-involvement-explained/5079786

http://www.abc.net.au/news/2013-11-08/the-chinese-embassy-bugging-controversy/5079148

http://www.news.com.au/national/australia-must-choose-between-chinese-cash-and-loyalty-to-the-us-as-se-asia-tensions-rise/story-fncynjr2-1227364070887

http://rt.com/news/270529-nsa-france-economy-wikileaks/ 

http://www.bloomberg.com/news/articles/2015-06-30/why-china-wants-a-strong-euro-as-greece-teeters

http://www.smh.com.au/federal-politics/political-news/china-not-fit-for-global-leadership-says-top-canberra-official-michael-thawley-20150630-gi1o1f.html 

- it makes sense that companies try to run lean rather than try to create. Everybody knows how to save. It's much more difficult to create something of value

- advertising is a broadcast means of achieving increased transactions but in spite of targeted advertising it is still incredibly inefficient. Based on previous experience even single digit click through rates for online advertising is considered suspect/possibly fraudulent http://adage.com/article/guest-columnists/study-advertising-half-effective-previously-thought/228409/

- the easiest way of estabishing the difference between what's needed and what's wanted is to turn off all advertising around you. Once you've done that, the difference between need and want becomes very strange and the efficacy of advertising on your perspective becomes much, much clearer

- most businesses fail. A lot of people basically have trouble running a business, have flawed business models, or don't achieve enough transactions to make it worthwhile

http://www.forbes.com/sites/ericwagner/2013/09/12/five-reasons-8-out-of-10-businesses-fail/

https://www.linkedin.com/pulse/20140915223641-170128193-what-are-the-real-small-business-survival-rates

http://www.smh.com.au/business/the-economy/google-says-give-rd-tax-breaks-to-small-techies-not-big-guys-20150407-1mfy30.html

http://smallbiztrends.com/2012/09/failure-rates-by-sector-the-real-numbers.html

http://www.isbdc.org/small-business-failure-rates-causes/

http://www.washingtonpost.com/blogs/fact-checker/wp/2014/01/27/do-9-out-of-10-new-businesses-fail-as-rand-paul-claims/

- immigration is a good thing provided that the people in question bring something to the economy. I look at the Japanese situation and wonder whether or not immigration is a more cost effective means of dealing with their ageing problem than 'Abenomics'. Even if all they do is re-patriate former nationals...

http://www.koreaherald.com/view.php?ud=20150628000326

- if you run through their numbers carefully, and think about where many of the world's top companies are headed, the performance (net profit in particular) of some of them aren't any where near impressive (percentage wise) as the share price growth in recent history. There are many small/mid cap firms that would out do them (% net profit wise) if you're looking to invest

http://www.gurufocus.com/financials/AAPL&affid=45223

https://finance.yahoo.com/q/ks?s=MSFT+Key+Statistics

http://www.marketwatch.com/investing/stock/amzn/financials

http://www.marketwatch.com/investing/stock/goog/financials

https://investor.google.com/financial/tables.html

- in software engineering people continually harp on about the benefits of Agile, Extreme programming and so on. Basically, all it is maintaining regular contact between staff members to get the best out of a piece of work. Peer pressure and continual oversight also forces you to remain productive. Think about this in the real world. The larger the teams are the more difficult it is to maintain oversight particuarly if the manager in question is of a poor standard and there are no systems in place to maintain standards. There is also a problem with unfettered belief in this metholodgy. If in general, the team members are unproductive or of a poor standard this will ripple throughout your team

- GDP is a horrible measure of productivity. As I've stated previously, the difference between perceived, effective, and actual value basically diguises where true value lies. Go spend some time in other parts of the world. I guarantee that there will be a massive difference in the way you view productivity (productivity means amount of work completed per unit time not overall work)

- a good measure of a person's productivity/value is what happens if they take a day off or a have a break. Observe, the increase in workload for each other staff member and how they deal with it

- people keep on harping on about self interest as the best way of maintaining productivity and encouraging people to work hard. However, I have a huge problem with this as it is incredibly hard to differentiate between actual, effective, and perceived value sometimes. At one particular firm, we had difficulties with this as well. I was therefore tasked with writing an application to monitor things (if you intend to write something along these lines please be mindful relevant HR and Surveillance laws in your jurisdiction. Also, keep the program 'silent'. Staff will likely alter their behaviour if they know that the program is running.). The funny thing is that even people you think are productive tend to work in bursts. The main difference is the amount of time that trasnpires between each piece of work and the rate of work that occurs during each burst. The other thing that you should know is that even with senior members of staff when you look at a lot of metrics it can be extremely difficult to justify their wage. Prepare to be surprised if you currently have poor oversight in your organisation. Lack of proper oversight breeds nepotism, lack of productivity, etc...

- you'll be shocked at what poor staff can do to your team. If the members in question is particularly bad he in effect takes a number of other staff out of the equation at the same time. Think about this. You all are recruited for highly skilled jobs but one team member is poor. If he continually has to rely on other staff then he in effects takes out another member of your team simultaneously (possibly more). Think about this when training new staff. Give them enough time/training to get a guage of what they'll be like but if they can't hold up their part of the deal be prepared to move them elsewhere within the organisation or let go of them. The same is also true in the opposite direction. Good employees have a multiplier effect. You'll only figure out the difference with proper oversight and monitoring. Without this, perceived value may completely throw you off

http://programmers.stackexchange.com/questions/179616/a-good-programmer-can-be-as-10x-times-more-productive-than-a-mediocre-one

http://swreflections.blogspot.com.au/2015/01/we-cant-measure-programmer-productivity.html

http://stackoverflow.com/questions/966800/mythical-man-month-10-lines-per-developer-day-how-close-on-large-projects - we like to focus in on large companies because they supposedly bring in a lot of business. The problem is if they have a monopoly. If they strangle the market of all value and don't put back in via taxes, employment, etc... the state in question could be in a lot of trouble down the line. If/when the company moves the economy would have evolved to see these companies as being a core component. Other surrouding will likely be poorly positioned to adapt when they leave for a place which offers better terms and/or conditions. The other problem is this, based on experience people are willing to except a lower wage to work for such firms (mostly for reasons of financial safety). There is no guarantee that you will be paid what you are worth

http://techcrunch.com/2015/06/28/policy-after-uber/

http://www.businessinsider.com/greeces-former-tax-collection-chief-harry-theoharis-explains-tax-evasion-problem-2015-7

http://www.irishtimes.com/business/economy/smes-account-for-99-7-of-business-enterprises-in-republic-1.2035800

http://www.irishtimes.com/business/economy/economy-primed-for-sustained-growth-says-goldman-sachs-1.2143071

http://www.wsj.com/articles/SB10001424127887324787004578496803472834948

http://www.afr.com/technology/technology-companies/ireland-scraps-google-tech-company-tax-breaks-20141019-119m80

https://en.wikipedia.org/wiki/Double_Irish_arrangement

http://blogs.cfainstitute.org/investor/2015/06/11/solutions-to-a-misbehaving-finance-industry/

http://www.theguardian.com/commentisfree/2015/jun/28/david-cameron-is-abusing-magna-carta-in-abolishing-our-rights

http://www.theguardian.com/world/2015/mar/25/irelands-economy-starting-to-fire-all-cylinders-imf-report

http://www.irishtimes.com/business/economy/who-owes-more-money-the-irish-or-the-greeks-1.2236034

http://www.theguardian.com/us-news/2015/feb/02/barack-obama-tax-profits-president-budget-offshore

http://www.smh.com.au/business/multinationals-channel-more-money-through-hubs-in-singapore-switzerland-than-ever-before-tax-office-says-20150204-1363u5.html

http://www.smh.com.au/business/retail/jeff-kennett-tells-coles-to-pay-12m-to-suppliers-20150630-gi19wv.html 

- when and if a large company collapses or moves the problem is the number of others who rely on it for business

- people keep on saying that there are safe industries from off shoring and automation. I think they're naive or haven't spent enough time around good technologists. Good employees will try to automate or develop processes to get things done more efficiently. Virtually all industries (or vast chunks of them) can be automated fully given time (trust me on this. I like to read a lot...).

http://www.technologyreview.com/view/519241/report-suggests-nearly-half-of-us-jobs-are-vulnerable-to-computerization/

http://www.futuristspeaker.com/2012/02/2-billion-jobs-to-disappear-by-2030/

http://www.forbes.com/sites/jmaureenhenderson/2012/08/30/careers-are-dead-welcome-to-your-low-wage-temp-work-future/

http://theconversation.com/australia-must-prepare-for-massive-job-losses-due-to-automation-43321

http://www.theguardian.com/business/2015/jun/16/computers-could-replace-five-million-australian-jobs-within-two-decades

Only way to keep yourself safe is to be multi-skilled and entrepreneurial or else extremely skilled at a particular profession. Even then there's no guarantee that you'll be safe

http://time.com/3938678/obamacare-supreme-court-uber/

http://techcrunch.com/2015/06/28/policy-after-uber/

- sometimes I think people just don't get it. A small number of outliers is all it takes in order to change group behaviour. Even if we ban regulate/automation there will be those who adopt it without any misgivings much like organised crime, and use of illegal migrants, cash economy, etc... Only real way is to force a cashless society so that we can run algorithms to check for unusual behaviour and breed a more puritan society

- minimal but effective regulation helps to level out the playing field. Making it too complex creates possible avenues for loopholes to be exploited. Too simple and without enough coverage and you have the same problem

- obvious ways to make sustained, long term money include creating something that others need or want, else have the ability to be able to change perception, to be able to see changes and adapt, arbitrage, and using a broadcast structure

- personal experience and history of others with emerging markets such as Asia and Africa says that results can be extremely variable. Without on the ground knowledge and oversight you can just as easily make a substantial profit as a massive loss through fraud. There is very little you can do about this about from taking due diligence and having measures/knowledge to be able to deal with it should it actually occur

http://timesofindia.indiatimes.com/world/uk/India-UKs-3rd-largest-job-creator-in-2014/articleshow/47714406.cms

- in reality, very few have a genuine chance of making it 'big', "Americans raised at the top and bottom of the income ladder are likely to remain there themselves as adults. Forty-three percent of those who start in the bottom are stuck there as adults, and 70 percent remain below the middle quintile. Only 4 percent of adults raised in the bottom make it all the way to the top, showing that the "rags-to-riches" story is more often found in Hollywood than in reality."

http://www.forbes.com/sites/jmaureenhenderson/2012/08/30/careers-are-dead-welcome-to-your-low-wage-temp-work-future/

- use first mover advantage as quickly as you can but have defensive measures in place

http://www.news.com.au/finance/business/is-the-free-ride-over-for-uber/story-fnda1bsz-1227419310284

- investment from third parties (angel investment, venture capital, etc...) can vary drastically. More and more want a guaranteed return on investment at least though

- based on what I've experienced VC is much more difficult to get locally than in Europe or the United States. Luckily, more companies are willing to invest provided you are posting good numbers. One other thing I've discovered locally is that they are too lazy/unwilling to help even if the idea/s may be good though (though this is changing)

http://www.afr.com/business/health/pharmaceuticals/merck-ceo-ken-frazier-on-keytruda-and-why-australians-miss-out-on-new-drugs-20150628-ghyisc

- we don't want to live day by day or have creditors/shareholders to report to so seek the highest profit whenever possible

- you can select a lot of numbers and prove essentially anything in business but their are certain numbers that you simply can't ignore such as net profit/income

- pay a person with cash by the hour where he has to do the numbers versus lump sump and he will look at things very differently. That goes for any profession, even high earning ones

- growth is great but only if it can be sustained and it is genuine. If you have susbtantial variation in growth such as having a few fantastic years of growth and then a sudden drop off that is fed by massive debt you could be in a bit of trouble. You may say that you can just sell off assets. If the growth wasn't good enough then do you see a problem? Moreover, what if you don't have something that it considered worthwhile or easy to sell off? For a state/business, your credit risk suddenly shoots up and you may possibly be priced out of the market. Targeted, sustainable, growth should be the target not growth at all costs. The Chinese position towards economic management is actually making a lot more sense to me now though I'm not certain that it would work quite as easily or be accepted in other states. You may say that we'll invest during good times? The problem is that we're often not wise enough to know when and where to invest

http://www.businessinsider.com/krugman-europe-greece-2015-6

http://www.businessinsider.com/el-erian-on-how-greece-will-impact-markets-2015-6

http://www.dawn.com/news/1162195/putins-next-challenge-propping-up-russias-troubled-banks

- in many places you are seeing a rise of left wing parties. The worrying thing is that they'll lose sight of the benefits of capitalism and fall into the trap of a more puritan communism/socialist system which hasn't really worked over the long term in the past. The other thing to be concerned about is that a lot of them don't have solid policies or answers to the problems which currently face us

http://theconversation.com/postcard-from-spain-where-now-for-the-quiet-revolution-43779

http://blogs.channel4.com/paul-mason-blog/greece-referendum-euro-die/3978

- if more people could distinguish real value from perceived and effective value, needs and wants, we would have less assetts bubbles and price gouging across the board

http://www.news.com.au/finance/real-estate/bis-shrapnel-report-reveals-property-prices-to-fall/story-fncq3era-1227416605503?from=google_rss&google_editors_picks=true

https://www.ozbargain.com.au/node/104348

http://www.news.com.au/world/breaking-news/nz-govt-slammed-over-10m-ny-apartment/story-e6frfkui-1227416038766

- there will be those who say who cares about the collective. Capitalism is composed of boom and bust cycles. Here's the problem. Most companies require debt to survive. If they can't survive that bust cycle they will be part of a collective collapse in the economy. Moreover, based on information I've come across other developed countries have looked at the plans for the Eurozone and the ways of dealing with high debt and are basically using that as the blueprint for the future. Your assets can and will be raided in the event of the state or systemic entities getting into trouble

http://www.heraldsun.com.au/business/greeks-stashing-money-in-homes-as-deadline-looms-for-debt-repayments/story-fni0d2cj-1227403214181?from=google_rss&google_editors_picks=true

http://www.nytimes.com/2015/06/30/business/dealbook/the-hard-line-on-greece.html?_r=0

http://www.usatoday.com/story/news/2015/06/29/evening-news-roundup-monday/29466899/

http://www.bbc.co.uk/news/world-europe-33324363

http://www.washingtonpost.com/blogs/wonkblog/wp/2015/06/30/7-questions-about-greeces-huge-crisis-you-were-too-embarrassed-to-ask/ 

- people say that we should get educated in order to have a high paying job but the problem is that we are increasingly siloed into specific roles. If we can't use the knowledge, the time and money we've spent on education has been for nothing. We require better organisation between educational curriculums and professional settings

http://www.financialexpress.com/article/companies/infosys-wipro-tech-mahindra-it-giants-revamp-culture-to-attract-young-talent-battle-start-ups/86718/

- even if governments are aware that there are problems that are cropping up with our version of capitalism, it's possible that there are those that may be saying that we have no choice but to keep the cycle going. It's the best of the worst

http://www.bbc.co.uk/news/world-europe-33303105

- globlisation essentially buys us more time before things come to a head (if they do). Most of the sceanarios point to organised debt forgiveness as a means of dealing with the problem. Private asset seizure is something that is being metioned everywhere. Raw commodities stored at secure locations may be your only source of safety if things look bad if you are a private citizen

http://www.washingtonpost.com/blogs/wonkblog/wp/2015/06/29/greece/

http://www.news.com.au/finance/small-business/those-selling-safes-are-cashing-in-on-greeces-financial-uncertainty/story-fn9evb64-1227422325045

http://www.news.com.au/finance/economy/what-a-grexit-would-look-like/story-e6frflo9-1227422412614

http://www.telegraph.co.uk/finance/economics/11712098/Europe-has-suffered-a-reputational-catastrophe-in-Greece.html 

- if you want a resilient economy you need maintain a level playing field, flexible workforce, and possibly limit the size and influence of major companies in your economy

http://www.vice.com/en_uk/read/the-irish-emigration-crisis--a-new-century-an-old-problem

- I don't get it. Heaps of countries have adequate blocking technology to be help deal with this if they deem it illegal. Deploy it correctly and your rioting problem is over with...

http://www.arkansasonline.com/news/2015/jun/27/hollande-uber-unit-illegal-dismantle-it/

http://www.theguardian.com/technology/2015/jun/26/uber-expansion-meets-global-revolt-and-crackdown

http://timesofindia.indiatimes.com/tech/tech-news/Officials-hint-at-possible-win-for-Uber-in-Mexico-City/articleshow/47861342.cms

- as stated previously, I've come to the conclusion that a lot of financial instruments are useless. They effectively provide a means of making money under any conditions. If we remove these instruments from play then I think that it may be possible that we may return to less speculative markets that depend more on fundamentals

- anyone can create something of value. The issue is whether it is negligible versus tangible value. This will also determine your business model

- you may know that ther is a bubble but as China and local experiences have demonstrated popping it gracefully is far from easy. Moreover, by the time you figure out there's a bubble it may often too late. Too many people may have too many vested interests

http://www.reuters.com/article/2015/06/29/us-usa-puertorico-restructuring-idUSKCN0P903Q20150629

http://www.businessinsider.com/puerto-rico-is-struggling-to-repay-its-debt-2015-6

http://jamaica-gleaner.com/article/commentary/20150630/editorial-jamaica-no-greece

http://www.heraldsun.com.au/business/breaking-news/world-bank-warns-china-on-reforms/story-fnn9c0hb-1227423791391?nk=5438df4578f2af3f2d269863d041c50c-1435746465 

- theory helps but you won't figure out how market economies work without first hand experience

http://www.afr.com/news/policy/budget/big-government-flourishes-under-tony-abbott-and-joe-hockey-20150513-gh0sgrhttp://www.dailytelegraph.com.au/news/nsw/joe-blasts-welfare-rich-who-have-more-money-to-spend-than-workers/story-fni0cx12-1227357517141

http://www.smh.com.au/business/australiachina-free-trade-agreement-favours-chinese-investors-20150621-ghthjr.html

http://www.afr.com/technology/telstra-cuts-broadband-plan-fees-to-counter-rivals-20150626-ghyir7

http://www.afr.com/opinion/columnists/trophy-trade-deals-wont-change-the-imfs-dismal-outlook-20150628-ghysnn

http://www.brisbanetimes.com.au/act-news/uberx-australian-drivers-working-as-coequals-to-rideshare-tech-company-20150629-ghvjx1.html

http://www.dailytelegraph.com.au/business/breaking-news/hockey-flying-blind-on-negative-gearing/story-fnn9c0gv-1227417798217?nk=0b226f408634f8d8ba57220c3d074f55-1435471944

http://www.abc.net.au/news/2013-11-08/the-chinese-embassy-bugging-controversy/5079148

http://www.macleans.ca/news/world/why-refugees-are-fleeing-france-for-britain/

http://www.businessinsider.com.au/facebooks-shot-at-cisco-just-got-deadly-2015-3

http://www.theglobeandmail.com/globe-drive/culture/technology/gyroscopes-will-allow-bike-to-stay-upright-when-stopped/article24920123/

http://www.businessinsider.in/5-things-Elon-Musk-believed-would-change-the-future-of-humanity-in-1995/articleshow/46831594.cms

Craige McWhirter: Craige McWhirter: How To Delete a Cinder Snapshot with a Status of error or error_deleting With Ceph Block Storage

Mon, 2015-06-29 18:29

When deleting a volume snapshot in OpenStack you may sometimes get an error message stating that Cinder was unable to delete the snapshot.

There are a number of reasons why a snapshot may be reported by Ceph as unable to be deleted, however the most common reason in my experience has been that a Cinder client connection has not yet been closed, possibly because a client crashed.

If you were to look at the snapshots in Cinder, the status is usually error or error_deleting:

% cinder snapshot-list +--------------------------------------+--------------------------------------+----------------+------------------------------------------------------------------+------+ | ID | Volume ID | Status | Display Name | Size | +--------------------------------------+--------------------------------------+----------------+------------------------------------------------------------------+------+ | 07d75992-bf3f-4c9c-ab4e-efccdfc2fe02 | 3004d6e9-7934-4c95-b3ee-35a69f236e46 | error | snappy:3004d6e9-7934-4c95-b3ee-35a69f236e46:2015-06-26T14:00:02Z | 40 | | 2db84ec7-6e1a-41f8-9dc9-1dc14e6ecef0 | 3004d6e9-7934-4c95-b3ee-35a69f236e46 | error_deleting | snappy:3004d6e9-7934-4c95-b3ee-35a69f236e46:2015-05-18T00:00:01Z | 40 | | 47fbbfe8-643c-4711-a066-36f247632339 | 3004d6e9-7934-4c95-b3ee-35a69f236e46 | available | snappy:3004d6e9-7934-4c95-b3ee-35a69f236e46:2015-06-29T03:00:14Z | 40 | | 52c43ec8-e713-4f87-b329-3c681a3d31f2 | 3004d6e9-7934-4c95-b3ee-35a69f236e46 | error_deleting | snappy:3004d6e9-7934-4c95-b3ee-35a69f236e46:2015-06-24T14:00:02Z | 40 | | a595180f-d5c5-4c4b-a18c-ca56561f36cc | 3004d6e9-7934-4c95-b3ee-35a69f236e46 | error | snappy:3004d6e9-7934-4c95-b3ee-35a69f236e46:2015-06-25T14:00:02Z | 40 | +--------------------------------------+--------------------------------------+----------------+------------------------------------------------------------------+------+

When you check Ceph you may find the following snapshot list:

# rbd snap ls my.pool.cinder.block/volume-3004d6e9-7934-4c95-b3ee-35a69f236e46 SNAPID NAME SIZE 2069 snapshot-2db84ec7-6e1a-41f8-9dc9-1dc14e6ecef0 40960 MB 2526 snapshot-52c43ec8-e713-4f87-b329-3c681a3d31f2 40960 MB 2558 snapshot-47fbbfe8-643c-4711-a066-36f247632339 40960 MB

The astute will notice that there are only 3 snapshots listed in Ceph yet 5 listed in Cinder. We can immediately exclude 47fbbfe8 which is available in both Cinder and Ceph, so there's no issues there.

You will also notice that the snapshots with the status error are not in Ceph and the two with error_deleting are. My take on this is that for the status error, Cinder never received the message from Ceph stating that this had been deleted successfully. Whereas for the status error_deleting status, Cinder had been unsuccessful in offloading the request to Ceph.

Each status will need to be handled separately , I'm going to start with the error_deleting snapshots, which are still present in both Cinder and Ceph.

In MariaDB, set the status from error_deleting to available:

MariaDB [cinder]> update snapshots set status='available' where id = '2db84ec7-6e1a-41f8-9dc9-1dc14e6ecef0'; Query OK, 1 row affected (0.00 sec) Rows matched: 1 Changed: 1 Warnings: 0 MariaDB [cinder]> update snapshots set status='available' where id = '52c43ec8-e713-4f87-b329-3c681a3d31f2'; Query OK, 1 row affected (0.00 sec) Rows matched: 1 Changed: 1 Warnings: 0

Check in Cinder that the status of these snapshots has been updated successfully:

% cinder snapshot-list +--------------------------------------+--------------------------------------+----------------+------------------------------------------------------------------+------+ | ID | Volume ID | Status | Display Name | Size | +--------------------------------------+--------------------------------------+----------------+------------------------------------------------------------------+------+ | 07d75992-bf3f-4c9c-ab4e-efccdfc2fe02 | 3004d6e9-7934-4c95-b3ee-35a69f236e46 | error | tuttle:3004d6e9-7934-4c95-b3ee-35a69f236e46:2015-06-26T14:00:02Z | 40 | | 2db84ec7-6e1a-41f8-9dc9-1dc14e6ecef0 | 3004d6e9-7934-4c95-b3ee-35a69f236e46 | available | tuttle:3004d6e9-7934-4c95-b3ee-35a69f236e46:2015-05-18T00:00:01Z | 40 | | 47fbbfe8-643c-4711-a066-36f247632339 | 3004d6e9-7934-4c95-b3ee-35a69f236e46 | available | tuttle:3004d6e9-7934-4c95-b3ee-35a69f236e46:2015-06-29T03:00:14Z | 40 | | 52c43ec8-e713-4f87-b329-3c681a3d31f2 | 3004d6e9-7934-4c95-b3ee-35a69f236e46 | available | tuttle:3004d6e9-7934-4c95-b3ee-35a69f236e46:2015-06-24T14:00:02Z | 40 | | a595180f-d5c5-4c4b-a18c-ca56561f36cc | 3004d6e9-7934-4c95-b3ee-35a69f236e46 | error | tuttle:3004d6e9-7934-4c95-b3ee-35a69f236e46:2015-06-25T14:00:02Z | 40 | +--------------------------------------+--------------------------------------+----------------+------------------------------------------------------------------+------+

Delete the newly available snapshots from Cinder:

% cinder snapshot-delete 2db84ec7-6e1a-41f8-9dc9-1dc14e6ecef0 % cinder snapshot-delete 52c43ec8-e713-4f87-b329-3c681a3d31f2

Then check the results in Cinder and Ceph:

% cinder snapshot-list +--------------------------------------+--------------------------------------+----------------+------------------------------------------------------------------+------+ | ID | Volume ID | Status | Display Name | Size | +--------------------------------------+--------------------------------------+----------------+------------------------------------------------------------------+------+ | 07d75992-bf3f-4c9c-ab4e-efccdfc2fe02 | 3004d6e9-7934-4c95-b3ee-35a69f236e46 | error | tuttle:3004d6e9-7934-4c95-b3ee-35a69f236e46:2015-06-26T14:00:02Z | 40 | | 47fbbfe8-643c-4711-a066-36f247632339 | 3004d6e9-7934-4c95-b3ee-35a69f236e46 | available | tuttle:3004d6e9-7934-4c95-b3ee-35a69f236e46:2015-06-29T03:00:14Z | 40 | | a595180f-d5c5-4c4b-a18c-ca56561f36cc | 3004d6e9-7934-4c95-b3ee-35a69f236e46 | error | tuttle:3004d6e9-7934-4c95-b3ee-35a69f236e46:2015-06-25T14:00:02Z | 40 | +--------------------------------------+--------------------------------------+----------------+------------------------------------------------------------------+------+ # rbd snap ls my.pool.cinder.block/volume-3004d6e9-7934-4c95-b3ee-35a69f236e46 SNAPID NAME SIZE 2558 snapshot-47fbbfe8-643c-4711-a066-36f247632339 40960 MB

So we are done with Ceph now, as the error snapshots do not exist there. As they only exist in Cinder, we need to mark them as deleted in the Cinder database:

MariaDB [cinder]> update snapshots set status='deleted', deleted='1' where id = '07d75992-bf3f-4c9c-ab4e-efccdfc2fe02'; Query OK, 1 row affected (0.00 sec) Rows matched: 1 Changed: 1 Warnings: 0 MariaDB [cinder]> update snapshots set status='deleted', deleted='1' where id = 'a595180f-d5c5-4c4b-a18c-ca56561f36cc'; Query OK, 1 row affected (0.00 sec) Rows matched: 1 Changed: 1 Warnings: 0

Now check the status in Cinder:

% cinder snapshot-list +--------------------------------------+--------------------------------------+-----------+------------------------------------------------------------------+------+ | ID | Volume ID | Status | Display Name | Size | +--------------------------------------+--------------------------------------+-----------+------------------------------------------------------------------+------+ | 47fbbfe8-643c-4711-a066-36f247632339 | 3004d6e9-7934-4c95-b3ee-35a69f236e46 | available | tuttle:3004d6e9-7934-4c95-b3ee-35a69f236e46:2015-06-29T03:00:14Z | 40 | +--------------------------------------+--------------------------------------+-----------+------------------------------------------------------------------+------+

Now your errant Cinder snapshots have been removed.

Enjoy :-)

Sridhar Dhanapalan: Twitter posts: 2015-06-22 to 2015-06-28

Mon, 2015-06-29 00:27

Russell Coker: RAID Pain

Sun, 2015-06-28 21:26

One of my clients has a NAS device. Last week they tried to do what should have been a routine RAID operation, they added a new larger disk as a hot-spare and told the RAID array to replace one of the active disks with the hot-spare. The aim was to replace the disks one at a time to grow the array. But one of the other disks had an error during the rebuild and things fell apart.

I was called in after the NAS had been rebooted when it was refusing to recognise the RAID. The first thing that occurred to me is that maybe RAID-5 isn’t a good choice for the RAID. While it’s theoretically possible for a RAID rebuild to not fail in such a situation (the data that couldn’t be read from the disk with an error could have been regenerated from the disk that was being replaced) it seems that the RAID implementation in question couldn’t do it. As the NAS is running Linux I presume that at least older versions of Linux have the same problem. Of course if you have a RAID array that has 7 disks running RAID-6 with a hot-spare then you only get the capacity of 4 disks. But RAID-6 with no hot-spare should be at least as reliable as RAID-5 with a hot-spare.

Whenever you recover from disk problems the first thing you want to do is to make a read-only copy of the data. Then you can’t make things worse. This is a problem when you are dealing with 7 disks, fortunately they were only 3TB disks and only each had 2TB in use. So I found some space on a ZFS pool and bought a few 6TB disks which I formatted as BTRFS filesystems. For this task I only wanted filesystems that support snapshots so I could work on snapshots not on the original copy.

I expect that at some future time I will be called in when an array of 6+ disks of the largest available size fails. This will be a more difficult problem to solve as I don’t own any system that can handle so many disks.

I copied a few of the disks to a ZFS filesystem on a Dell PowerEdge T110 running kernel 3.2.68. Unfortunately that system seems to have a problem with USB, when copying from 4 disks at once each disk was reading about 10MB/s and when copying from 3 disks each disk was reading about 13MB/s. It seems that the system has an aggregate USB bandwidth of 40MB/s – slightly greater than USB 2.0 speed. This made the process take longer than expected.

One of the disks had a read error, this was presumably the cause of the original RAID failure. dd has the option conv=noerror to make it continue after a read error. This initially seemed good but the resulting file was smaller than the source partition. It seems that conv=noerror doesn’t seek the output file to maintain input and output alignment. If I had a hard drive filled with plain ASCII that MIGHT even be useful, but for a filesystem image it’s worse than useless. The only option was to repeatedly run dd with matching skip and seek options incrementing by 1K until it had passed the section with errors.

for n in /dev/loop[0-6] ; do echo $n ; mdadm –examine -v -v –scan $n|grep Events ; done

Once I had all the images I had to assemble them. The Linux Software RAID didn’t like the array because not all the devices had the same event count. The way Linux Software RAID (and probably most RAID implementations) work is that each member of the array has an event counter that is incremented when disks are added, removed, and when data is written. If there is an error then after a reboot only disks with matching event counts will be used. The above command shows the Events count for all the disks.

Fortunately different event numbers aren’t going to stop us. After assembling the array (which failed to run) I ran “mdadm -R /dev/md1” which kicked some members out. I then added them back manually and forced the array to run. Unfortunately attempts to write to the array failed (presumably due to mismatched event counts).

Now my next problem is that I can make a 10TB degraded RAID-5 array which is read-only but I can’t mount the XFS filesystem because XFS wants to replay the journal. So my next step is to buy another 2*6TB disks to make a RAID-0 array to contain an image of that XFS filesystem.

Finally backups are a really good thing…

Related posts:

  1. RAID and Bus Bandwidth As correctly pointed out by cmot [1] my previous post...
  2. Some RAID Issues I just read an interesting paper titled An Analysis of...
  3. Reliability of RAID ZDNet has an insightful article by Robin Harris predicting the...

Joshua Hesketh: git.openstack.org adventures

Sat, 2015-06-27 15:25

Over the past few months I started to notice occasional issues when cloning repositories (particularly nova) from git.openstack.org.

It would fail with something like

git clone -vvv git://git.openstack.org/openstack/nova . fatal: The remote end hung up unexpectedly fatal: early EOF fatal: index-pack failed

The problem would occur sporadically during our 3rd party CI runs causing them to fail. Initially these went somewhat ignored as rechecks on the jobs would succeed and the world would be shiny again. However, as they became more prominent the issue needed to be addressed.

When a patch merges in gerrit it is replicated out to 5 different cgit backends (git0[1-5].openstack.org). These are then balanced by two HAProxy frontends which are on a simple DNS round-robin.

+-------------------+ | git.openstack.org | | (DNS Lookup) | +--+-------------+--+ | | +--------+ +--------+ | A records | +-------------------v----+ +-----v------------------+ | git-fe01.openstack.org | | git-fe02.openstack.org | | (HAProxy frontend) | | (HAProxy frontend) | +-----------+------------+ +------------+-----------+ | | +-----+ +---+ | | +-----v------------------------------------v-----+ | +---------------------+ (source algorithm) | | | git01.openstack.org | | | | +---------------------+ | | +---| git02.openstack.org | | | | +---------------------+ | | +---| git03.openstack.org | | | | +---------------------+ | | +---| git04.openstack.org | | | | +---------------------+ | | +---| git05.openstack.org | | | | (HAProxy backend) | | | +---------------------+ | +------------------------------------------------+

Reproducing the problem was difficult. At first I was unable to reproduce locally, or even on an isolated turbo-hipster run. Since the problem appeared to be specific to our 3rd party tests (little evidence of it in 1st party runs) I started by adding extra debugging output to git.

We were originally cloning repositories via the git:// protocol. The debugging information was unfortunately limited and provided no useful diagnosis. Switching to https allowed for more CURL output (when using GIT_CURL_VERBVOSE=1 and GIT_TRACE=1) but this in itself just created noise. It actually took me a few days to remember that the servers are running arbitrary code anyway (a side effect of testing) and therefore cloning from the potentially insecure http protocol didn’t provide any further risk.

Over http we got a little more information, but still nothing that was conclusive at this point:

git clone -vvv http://git.openstack.org/openstack/nova . error: RPC failed; result=18, HTTP code = 200 fatal: The remote end hung up unexpectedly fatal: protocol error: bad pack header

After a bit it became more apparent that the problems would occur mostly during high (patch) traffic times. That is, when a lot of tests need to be queued. This lead me to think that either the network turbo-hipster was on was flaky when doing multiple git clones in parallel or the git servers were flaky. The lack of similar upstream failures lead me to initially think it was the former. In order to reproduce I decided to use Ansible to do multiple clones of repositories and see if that would uncover the problem. If needed I would have then extended this to orchestrating other parts of turbo-hipster in case the problem was systemic of something else.

Firstly I need to clone from a bunch of different servers at once to simulate the network failures more closely (rather than doing multiple clones on the one machine or from the one IP in containers for example). To simplify this I decided to learn some Ansible to launch a bunch of nodes on Rackspace (instead of doing it by hand).

Using the pyrax module I put together a crude playbook to launch a bunch of servers. There is likely much neater and better ways of doing this, but it suited my needs. The playbook takes care of placing appropriate sshkeys so I could continue to use them later.

--- - name: Create VMs hosts: localhost vars: ssh_known_hosts_command: "ssh-keyscan -H -T 10" ssh_known_hosts_file: "/root/.ssh/known_hosts" tasks: - name: Provision a set of instances local_action: module: rax name: "josh-testing-ansible" flavor: "4" image: "Ubuntu 12.04 LTS (Precise Pangolin) (PVHVM)" region: "DFW" count: "15" group: "raxhosts" wait: yes register: raxcreate - name: Add the instances we created (by public IP) to the group 'raxhosts' local_action: module: add_host hostname: "{{ item.name }}" ansible_ssh_host: "{{ item.rax_accessipv4 }}" ansible_ssh_pass: "{{ item.rax_adminpass }}" groupname: raxhosts with_items: raxcreate.success when: raxcreate.action == 'create' - name: Sleep to give time for the instances to start ssh #there is almost certainly a better way of doing this pause: seconds=30 - name: Scan the host key shell: "{{ ssh_known_hosts_command}} {{ item.rax_accessipv4 }} >> {{ ssh_known_hosts_file }}" with_items: raxcreate.success when: raxcreate.action == 'create' - name: Set up sshkeys hosts: raxhosts tasks: - name: Push root's pubkey authorized_key: user=root key="{{ lookup('file', '/root/.ssh/id_rsa.pub') }}"

From here I can use Ansible to work on those servers using the rax inventory. This allows me to address any nodes within my tenant and then log into them with the seeded sshkey.

The next step of course was to run tests. Firstly I just wanted to reproduce the issue, so in order to do that it would crudely set up an environment where it can simply clone nova multiple times.

--- - name: Prepare servers for git testing hosts: josh-testing-ansible* serial: "100%" tasks: - name: Install git apt: name=git state=present update_cache=yes - name: remove nova if it is already cloned shell: 'rm -rf nova' - name: Clone nova and monitor tcpdump hosts: josh-testing-ansible* serial: "100%" tasks: - name: Clone nova shell: "git clone http://git.openstack.org/openstack/nova"

By default Ansible runs with 5 folked processes. Meaning that Ansible would work on 5 servers at a time. We want to exercise git heavily (in the same way turbo-hipster does) so we use the –forks param to run the clone on all the servers at once. The plan was to keep launching servers until the error reared its head from the load.

To my surprise this happened with very few nodes (less than 15, but I left that as my minimum testing). To confirm I also ran the tests after launching further nodes to see it fail at 50 and 100 concurrent clones. It turned out that the more I cloned the higher the failure rate percentage was.

Now that I had the problem reproducing, it was time to do some debugging. I modified the playbook to capture tcpdump information during the clone. Initially git was cloning over IPv6 so I turned that off on the nodes to force IPv4 (just in case it was a v6 issue, but the problem did present itself on both networks). I also locked git.openstack.org to one IP rather than randomly hitting both front ends.

--- - name: Prepare servers for git testing hosts: josh-testing-ansible* serial: "100%" tasks: - name: Install git apt: name=git state=present update_cache=yes - name: remove nova if it is already cloned shell: 'rm -rf nova' - name: Clone nova and monitor tcpdump hosts: josh-testing-ansible* serial: "100%" vars: cap_file: tcpdump_{{ ansible_hostname }}_{{ ansible_date_time['epoch'] }}.cap tasks: - name: Disable ipv6 1/3 sysctl: name="net.ipv6.conf.all.disable_ipv6" value=1 sysctl_set=yes - name: Disable ipv6 2/3 sysctl: name="net.ipv6.conf.default.disable_ipv6" value=1 sysctl_set=yes - name: Disable ipv6 3/3 sysctl: name="net.ipv6.conf.lo.disable_ipv6" value=1 sysctl_set=yes - name: Restart networking service: name=networking state=restarted - name: Lock git.o.o to one host lineinfile: dest=/etc/hosts line='23.253.252.15 git.openstack.org' state=present - name: start tcpdump command: "/usr/sbin/tcpdump -i eth0 -nnvvS -w /tmp/{{ cap_file }}" async: 6000000 poll: 0 - name: Clone nova shell: "git clone http://git.openstack.org/openstack/nova" #shell: "git clone http://github.com/openstack/nova" ignore_errors: yes - name: kill tcpdump command: "/usr/bin/pkill tcpdump" - name: compress capture file command: "gzip {{ cap_file }} chdir=/tmp" - name: grab captured file fetch: src=/tmp/{{ cap_file }}.gz dest=/var/www/ flat=yes

This gave us a bunch of compressed capture files that I was then able to seek the help of my colleagues to debug (a particular thanks to Angus Lees). The results from an early run can be seen here: http://119.9.51.216/old/run1/

Gus determined that the problem was due to a RST packet coming from the source at roughly 60 seconds. This indicated it was likely we were hitting a timeout at the server or a firewall during the git-upload-pack of the clone.

The solution turned out to be rather straight forward. The git-upload-pack had simply grown too large and would timeout depending on the load on the servers. There was a timeout in apache as well as the HAProxy config for both frontend and backend responsiveness. The relative patches can be found at https://review.openstack.org/#/c/192490/ and https://review.openstack.org/#/c/192649/

While upping the timeout avoids the problem, certain projects are clearly pushing the infrastructure to its limits. As such a few changes were made by the infrastructure team (in particular James Blair) to improve git.openstack.org’s responsiveness.

Firstly git.openstack.org is now a higher performance (30GB) instance. This is a large step up from the previous (8GB) instances that were used as the frontend previously. Moving to one frontend additionally meant the HAProxy algorithm could be changed to leastconn to help balance connections better (https://review.openstack.org/#/c/193838/).

+--------------------+ | git.openstack.org | | (HAProxy frontend) | +----------+---------+ | | +------------------------v------------------------+ | +---------------------+ (leastconn algorithm) | | | git01.openstack.org | | | | +---------------------+ | | +---| git02.openstack.org | | | | +---------------------+ | | +---| git03.openstack.org | | | | +---------------------+ | | +---| git04.openstack.org | | | | +---------------------+ | | +---| git05.openstack.org | | | | (HAProxy backend) | | | +---------------------+ | +-------------------------------------------------+

All that was left was to see if things had improved. I rerun the test across 15, 30 and then 45 servers. These were all able to clone nova reliably where they had previously been failing. I then upped it to 100 servers where the cloning began to fail again.

Post-fix logs for those interested:

http://119.9.51.216/run15/

http://119.9.51.216/run30/

http://119.9.51.216/run45/

http://119.9.51.216/run100/

http://119.9.51.216/run15per100/

At this point, however, I’m basically performing a Distributed Denial of Service attack against git. As such, while the servers aren’t immune to a DDoS the problem appears to be fixed.

Binh Nguyen: The Value of Money - Part 3

Sat, 2015-06-27 03:38
The Western world generally saw the collapse of the Soviet Union as proof positive of the superiority of capitalism over communism/socialism. Most of the arguments bordered along the lines that the sheer scale of managing an economy, that it resulted in nepotism, bred corruption, stifled innovation, and that it didn't feed into the needs and wants of it's constituents were the reasons for their failure. The irony is that you can see many of the same flaws in communism and socialism that you see in capitalism now. Given the fact that more and more developed economies are getting into trouble I wonder whether this is the true way forward. The European Union, United States, Japan, and others have all recently endured serious economic difficulty and have been projected to continue to experience prolonged issues.



My belief that if capitalism and free market economics is to work into the future constraints must be placed on the size of firms relative to the size of the market/economy. Below are some reasons for my belief in this as well as some other notes regarding market economics:- I believe that one of the reasons we only favour free market economics because it limits the severity of problems if/when someone/something collapses. If a government collapses you have trouble everywhere. If a company collapses it only impacts the company and the immediate supply chain, distributers, retailers, etc...

- the other problem is most of the companies that grow to this size have no choice but to be driven by greed. Even if they pay their fair share of taxes most of them rely on debt of some sort in order to maintain a viable business. Without cash flow from the stock market, their creditors, etc... they can't continue to pay the bills. Hence, they must satisfy their own needs as well as that of their shareholders and creditors at the expense of those in the wider community. An example of this are the large retail chains that operate in many of the more developed countries. The problem is that their power can now rival that of the state. For instance, in Australia, "Almost 40 cents in every dollar we spend at the shops is now taken by a Woolworths or Wesfarmers-owned retail entity" with their interests including the "interests in groceries, fuel, liquor, gambling, office supplies, electronics, general merchandise, insurance and hardware, sparking concerns that consumers will pay more."

http://www.news.com.au/finance/money/coles-and-woolworths-receive-almost-40-per-cent-of-australian-retail-spending/story-e6frfmd9-1226043866311

If the chain collapses it's likely that hundreds of thousands of jobs will be lost in the event of administration/receivership. I'm arguing that we need to spread the risk a bit. If one part collapses it doesn't bring the whole thing crashing down around you

http://www.instantshift.com/2010/02/03/22-largest-bankruptcies-in-world-history/

- despite politician's complaints about MNCs/TNCs not contributing their fair share towards the tax base they aren't willing to make enough of an effort to change things to create those circumstances. There needs to be an understanding that without someone to buy their products and services these companies will go bankrupt. Large firms need employees and consumers as much as we need their tax revenue

- the irony is that we believe that since companies are large they are automatically successful, we should support them. Think about many of the recent large defense programs that were undertaken by large firms. As indicated previously, there's currently no incentive for them to help the state. They just want to survive and generate profits. The JSF program was deliberately structured in such a way that we've ended up with a fighter jet that isn't up to the original design spec, well and truly over the desired budgetary parameters, and way beyond the original design constraints putting the national security of many allied nations at risk

https://medium.com/war-is-boring/test-pilot-admits-the-f-35-can-t-dogfight-cdb9d11a875

http://www.janes.com/article/52715/jpo-counters-media-report-that-f-35-cannot-dogfight

- progress within the context of market economics is often only facilitated through proper competition and regulation. At the moment, many of the largest donors towards political parties are large companies. This results in a significant distortion of the playing field and what the ultimate decision makers deem to be important issues.

http://www.nytimes.com/2015/06/25/business/obama-bolsters-his-leverage-with-trade-victory-but-at-a-cost.html?_r=0

https://en.wikipedia.org/wiki/Global_saving_glut

Think about nature of the pharmaceutical industry and electronics/IT industries. They both complain that progress (research and development) is difficult. The irony is that it's difficult to argue this if you're not making any worthwhile attempts at it. Both sectors sit atop enough savings to be able to cure much of the world's current woes but they have absolutely no incentive to bring it back on shore for it to be properly tax or to spend it

http://money.cnn.com/2015/03/20/investing/stocks-companies-record-cash-level-oil/

http://www.telegraph.co.uk/finance/11038180/Global-firms-sitting-on-7-trillion-war-chest.html

http://www.telegraph.co.uk/finance/budget/9150406/Budget-2012-UK-companies-are-sitting-on-billions-of-pounds-so-why-arent-they-spending-it.html

http://www.theguardian.com/commentisfree/2013/may/13/tax-havens-hidden-billions

Moreover, they more often than not just use their existing position to continue to exploit the market. A lot of electronics now is simply uneconomical or impossible to repair locally which means that you have to purchase new products once it has gone out of warranty and has failed due to engineered lifecycles (they are designed to fail after a particular period. If they didn't they would suffer the same fate that some car manufacturers have been complaining about. If they don't fail no one will buy new cars). My belief is that there should be tax concesssions if they are willing or they should be forced to invest into SME firms (which comprise the bulk of the economy) via secondary small capitalisation type funds (especially if the company doesn't know what to do with spare cash and it is left 'stagnant'). Ironically, returns on broad based funds in this area (longer term) more often than not exceed the growth of the company in question as well as the economy in general

- sometimes I wonder whether or not managing an economy (from a political perspective) is much the same as operating as a market analyst. You're effectively taking calculated bets on how the world will end up in the future. Is it possible that good economic managers need to be more lucky than skillful?

http://www.amazon.com/Random-Walk-Down-Wall-Street/dp/0393330338

https://en.wikipedia.org/wiki/A_Random_Walk_Down_Wall_Street

- in some cases, the nature of capitalism is such that the state has grown so large (because in general government services aren't profitable) that they are beginning to groan under the pressure that many of the more developed nations are now feeling. This is a case of both mis-management and a mis-understanding of how to use capitalism to your advantage

- one of the biggest contradictions in business is that it should all come down to the bottom line. The stupid aspect of this is that most companies have double digit turnover and continue to make the excuse that you should simply put up with whatever is thrown at you even if employee turnover is high. If workplaces were generally more civilised and conditions better then you would have a huge cost removed from your business (loss of employee, advertising, training, etc...)

- normally, when people are taught about life, we start with the small and simple examples and then we are pushed into more complex and advanced examples. The irony is that is often the opposite of the way we are taught about business. We are taught to dream big and win big or else crash and burn and learn your place in society. There is a major problem with this. In the Australian economy, SME business accounts for 96% of the economy. It is similar elsewhere. People leaving our educational institutions basically aren't equipped to be able to run make money by themselves right out of school. Help them/teach them how and you could help the overall economy as well as these students by equipping them to be able to look after their own needs reducing the burden on the social welfare system and giving them valuable employment experience that may be worthwhile later down the track. Most students are equipped to work for other people not to start their own company or operate as individuals

http://www.smartcompany.com.au/technology/information-technology/31806-number-of-businesses-in-australia-continues-to-stagnate-abs.html

- all politicians (and people in general) like to talk about the success of their country in being able to attract MNCs/TNCs to employ people locally. However, the problem is that they aren't the main employment drivers in the economy. Across most of the world's economies small businesses are the driving force ("Such firms comprise around 99% of all businesses in most economies and between half and three quarters of the value added. They also make a significant contribution to employment and are of interest to governments primarily for their potential to create more jobs."). One wonders that even with the increased business (direct and indirect) around a large firm when they exist in a country are you getting value for money (especially if you are subsidising their local existence)?

http://theconversation.com/growing-the-global-economy-through-small-to-medium-enterprise-the-g20-sme-conference-28307

- we actually do ourselves somewhat of a disservice by creating a perception that dreaming and living big is what you should want. Popular culture makes it feel like as though if you don't go to the right schools, work for the right companies, and so on you are a failure. The irony is that if every single graduate were taught about how to commercialise their their ideas while at school I believe that we would have a far more flexible, innovative, economy. Moreover, both they as well as economy in general would get a return on investment. It's no good telling people how to be enterpreneurial if they don't know how to be enterpreneurial.

- the irony of the large donor phenomenon is that SME business accounts for most of the activity within the economy...

http://www.smartcompany.com.au/technology/information-technology/31806-number-of-businesses-in-australia-continues-to-stagnate-abs.html

- as we've discussed previously on this blog the primary ways you can make money are to create something of value or by changing the perceived value of something such that people will want to buy it no matter what the disparity between perceived value versus effective value. Once upon a time I look at German prestige and performance vehicles to be the pinnacle or automative engineering. The more I've learned about them the less impressed I've become. If I told you the evidence points to them being the least reliable, the vehicles which depreciate the most (within any given time frame), most expensive to repair, the most expensive to insure and service, average safety, and that often only have comparable technology to other cars (once you cut through the marketing speak) you'd think that people would be incredibly stupid to purchase them. Yet, this trend continues...

http://usedcars.about.com/od/research/fl/10-Least-Reliable-Used-Car-Brands.htm

http://www.bbc.com/news/business-32332210

http://rac.com.au/motoring/motoring-advice/buying-a-car/running-costs

http://rac.com.au/news-community/road-safety-and-transport/safe-cars/how-safe-is-your-car/used-car-safety-ratings

Another good example is the upper echelons of professional sport and artistry (includes music, art, etc...). If anybody told you that you were paying several hundred dollars an hour to watch a group of individuals kick a ball you'd think that they were mad. The horrible part is when you realise top tier amateur competitions which are free to watch can be just as entertaining and skillful

- in reality, in the real world very very rarely are pure market forces at play and it often takes a lot of time for it to get through to them that for all the stuff/theory that you learn at school there's a lot more that you will also learn in the real world

- most industries fit into the following categories; something that you need or something that you want. By selling people a dream we can turn what you want into something you need and create employment from it

- if you want to make abnormal (excess) profits it's mainly about being able to distinguish between perceived, effective, and actual value. Once you can establish this you can exploit it. This is easier said than done though. Let's say you discovered Lionel Messi playing in the streets of Somalia versus Paris. More than likely, you'd value him much less if we were found in Somalia. Sometimes it can be pretty obvious, at other times it's not much different from predicting the future. For instance, the iPod was essentially a re-modled MP3 player with an integrated software solution/ecosystem, Coke is basically just a sweet, fizzy drink which is actually beaten by Pepsi in blind tests

- we like short term thinking because we like the notion that we can make a lot of money in a short space of time. That means that we can retire early, purchase luxury goods and services. The irony is that this feeds into a disparity between actual, perceived, and effective value which means that flawed businesses can continue to still work. The irony is that this flaw works in practice but in the long term it can results in asset bubbles. Valuation at the correct level is in collective's overall interests

- risk isn't necessarily directly related to reward if you're modelling is good. One way to reduce risk is to let others take it first. You might not make a massive name for yourself but should at least not break bank for a high risk project. This has been a common theme in the Russian and Chinese defense establishments where they have often taken significant cues from American technology

- it's becoming clearer to me that many financial instruments actually aren't required. The industry itself relies on the fact that that many will fall for the perceived notion that you can make a lot of money in a small amount of time or for little labour. However, the reality is that most will make a lot less that what is perceived to be the case. An example of this is the following. Many financial instruments are created for the express purpose of increasing risk exposure and therefore possible profits/losses. In reality, most people lose. It's like a casino where the house wins most the time. The other irony is the following, while liquidity can have a direct correlation with volatility (allows you to reach a more valid price earlier especially if many are involved in pricing), the same is also true in the opposite direction. It only takes a few minor outliers to be able to change the perception where value within the market exists

http://blogs.cfainstitute.org/investor/2015/06/11/solutions-to-a-misbehaving-finance-industry/

- may SME firms collapse within a short time frame but easy credit makes it easier for bad business models to continue to exist. The same is also true of the United States economy where uncompetitive industries were allowed to continue to exist for a long time without adequate trade barriers. If the barriers are lifted we should create circumstances where we force companies to alter their strategies earlier or force them to re-structure/collapse/declare bankruptcy. It will help to reduce the impact of when we provide credit to flawed companies which ultimately collapse

- the way we measure credit risk is often questionable. Financial institutions often turn away low income earners because they are considered a credit risk. I looked at this further and the rates that they are actually charged are diabolical. At one particular place, they were charging 10% for a one week loan, 15% for a 3 week loan, and then 25% for a month long loan. If the industry was so risky though how does it continue to exist? Most of the people who understand the problem have basically said that people who require this money simply have a hard time budgeting and managing their affairs. Essentially providing them with a lump sum component every once in a while makes them believe that they can spend freely. The irony is that the rest of society is also somewhat guilty of this. If we were paid cash and by the hour (rather than regular lump sum payments) and had to pay a component of our bills and other expenses each day we would look at our purchases very differently

http://www.news.com.au/finance/real-estate/stamp-duty-scandal-tony-abbott-under-pressure-to-scrap-our-worst-tax-amid-disastrous-poll/story-fndban6l-1227398035046?from=google_rss&google_editors_picks=true

http://www.perthnow.com.au/news/breaking-news/welfare-card-trial-sites-still-undecided/story-fnhrvfuw-1227398497797?nk=2dc00eb5accf0aef95bbb39faeb08ba0-1434358050

At the other end of the scale, there exists another paradox/contradiction. I've heard stories about people with relatively high incomes being denied credit even though their credit history was good (companies can't make money if you don't breach credit conditions every once in a while). Despite what we say about free market economics, regulatory frameworks, etc... the system is corrupt. It's just not as overt and no one likes/wants to admit it.

- despite what many may think of him, I think Vladamir Putin is actually trying to look after his country's best interests. The collapse of the Soviet Union gave rise to the oligarch. A circumstance that was facilitated by the nature of free market economics without an adequate framework (rules and regulations such as that provided by law). Essentially, the state was replaced by private enterprise where the needs of the many were placed lower on the pecking order than had the state still been in charge. I understand his perspective but I don't believe in the way he has gone about things

https://en.wikipedia.org/wiki/Revolutions_of_1989

https://en.wikipedia.org/wiki/Socialism

https://en.wikipedia.org/?title=Communism

https://en.wikipedia.org/wiki/Capitalism

http://www.msnbc.com/msnbc/pope-francis-rejects-communism-critique

http://ncronline.org/blogs/francis-chronicles/pope-francis-concern-poor-sign-gospel-not-red-flag-communism

http://www.marxist.com/kievs-contemporary-anti-communism-and-the-crimes-of-the-oligarchys-very-existence.htm

- people say that we should do more and spend more in the fight against organised crime. The stupid, ironic thing is that when society is unfair and unjust organised crime grows much stronger because it provides people with a way of making a living. In Europe, the Italian mafia has grown much stronger with the advent of the European economic difficulties and it was much the same in Japan when their asset bubble burst during the 90s

https://en.wikipedia.org/wiki/Lost_Decade_%28Japan%29

- the EU was borne of the fact that no one wanted war again in Europe. It feels like much the same with the rest of the world. We've used progress and better living conditions as an argument against going to war. However, the world has essentally ended up engaging in an effective 'Cold War'. Much of the world's spending revolves around the notion of deterrence. Namely, if I go to want to go to war with you I know that I'll suffer just as much damage (if not more)

https://en.wikipedia.org/wiki/List_of_countries_by_military_expenditures

http://www.globalissues.org/article/75/world-military-spending

There are a number of ways around this. By reaching a concensus for that countries will no longer attempt to project power outwards (defend yourself only, don't interfere with others. Highly unlikely.), invasion will no longer be part of the future landscape (other countries will come to the aide of those in trouble. Unlikely especially with the rise of terrorism.), or else collapse a economies such that countries will no longer be able to afford to spend on defense. The troubling thing is that the last scenario has actually been outlined in various US intelligence and defense reports. It's essentially war without war. If you can wreak havoc in someone's economy then they'll no longer be a problem for you. The irony is that the larger your intelligence apparatus the more likely you can engage in this style of activity. Previous leaked reports and WikiLeaks has made me highly skeptical that the average country doesn't engage in this style of activity.

http://www.theguardian.com/world/2013/aug/29/us-intelligence-spending-double-9-11-secret-budget

https://en.wikipedia.org/wiki/United_States_intelligence_budget

The irony is that you if you don't engage in these activities you may lose a significant advantage. If you do, you're sort of left to question whether or not you are the good guy in this affair

- people who haven't spent enough time in the real world only often understand the theory. Once you understand how things actually work your whole perspective changes. Let's take the housing asset/bubble that we may be going through. As stated previously, making abnormal profits is about managing the difference between perceieved, actual, and effective value. It's clear that in theory boosting supply may change things. The thing I've discovered is that in free market economics it only takes a small thing to change perception. Once the perception snow balls you're stuck with the same problem. This is the same whether it is a new home buyer or a foreign investor purchasing in the local market

http://www.smh.com.au/nsw/mike-bairds-400m-boost-for-infrastructure-fund-to-tackle-housing-affordability-crisis-20150621-ghtfr8.html

http://www.theglobeandmail.com/report-on-business/milk-surplus-forcing-canadas-dairy-industry-to-dump-supply/article25030753/

http://www.brisbanetimes.com.au/it-pro/rental-growth-slowdown-signals-residential-property-bust-on-the-way-20150626-ghxkdr

http://www.news.com.au/finance/real-estate/economists-claim-australia-in-midst-of-largest-housing-bubble-on-record/story-fncq3era-1227410053643?from=google_rss&google_editors_picks=true

- a business structure is simply a focal point of communication between business and consumer. It also affords the opportunity for a government to tax it more effectively

- by being so insistent on upskilling and education it makes low labour costs almost impossible to achieve. This makes a lot of infrastructure projects in developed countries impossible because they are economically unviable. A good example of this is 457 visas in Australia, and illegal immigration in the United States (especially from Mexico) which are often used and abused to acheive lower labour costs than otherwise would have been possible. Another example of this is the Snowy River Hydroelectricity project. It's said that hardly anyone on site knew English and that often people just learned on the job.

http://www.politico.com/story/2015/06/donald-trump-calls-jeb-bush-unhappy-119153.html?ml=ri

Another recent project put this into perspective. It was said that building a building infrastructure (tunnels, office blocks, etc...) in China, shipping it, and then assembling it here in Australia would be more cost effective then building it here alone. We need to give people a chance no matter what their education or skill level if we are to balance government budgets and to reduce the incidents of off-shoring without necessarily having to resort to often expensive anti-shoring techniques such as tarrifs, rebates, taxes, etc...

- our perception of success feels odd sometimes. If you look up the background of Rupert Murdoch, Donald Trump, and several others you'll see that thye are continually on the point of brankruptcy. Under normal circumstances anyone continually on verge of losing everything would be considered mediocre but in the business world they're considered successful because they can keep the whole thing going... Also, look at the poverty figures for the United States, Germany, United Kingdom, United Arab Emirates, Iran, and Japan. Notice the odd one out? Iran has been under sanction for a long time for their alleged nuclear research activities and yet the level of poverty in Iran is comparable to all these others.

https://en.wikipedia.org/wiki/List_of_countries_by_percentage_of_population_living_in_poverty

- the only other way to achieve lower costs in developed countries is to resort to automation and robots (else tap developing countries for lower priced components). I've looked at Australian car manufacturing plants and American and European plants for mass produced vehicles. The level of automation in American and European plants seem to be significantly higher with build quality that is comparable

http://www.kyodonews.net/news/2015/06/20/21340

http://forums.whirlpool.net.au/archive/2050953

- the perception is that we always should hire the best and brightest in order to get the job done and that we should try to do our best to make them happy. The irony is that I've worked on both sides of the fence. By hiring only the best and brightest (perceived to be. A lot of the time the best and brightest don't necessarily get hired based on what I've seen) and only settling on them we force wages up across the board and we make work more difficult for your existing workforce. It may even be more difficult to keep them happy. The other irony is that there are many wealthy global companies who can afford to hire away your best staff forcing prices up even further. Complete free trade works in favour of those who are already wealthy and makes it harder for those down the chain to make a living and to progress

- if all the best and brightest are hired by the same companies (based on personal experience) you aren't necessarily always going to get the best out of them. Companies have an increasing tendency (regulatory as well as political issues) to pigeon hole them into specific roles which doesn't allow them to realise their full and complete potential. The individual, company, as well as the collective lose out

http://blogs.cfainstitute.org/investor/2015/06/11/solutions-to-a-misbehaving-finance-industry/

- we believe in out current style of capitalism because we have a perception that it gives everyone a chance in life to be and do whatever they want. In reality, it's a lot more complicated. At it's very core I think it's very much like Winston Churchhill's opinion of the Westminster parlimentary system, "Democracy is the worst form of government, except for all the others."

http://www.goodreads.com/quotes/267224-democracy-is-the-worst-form-of-government-except-for-all

- it's clear that I believe in limited capitalism and for the most part we should try to work with those within our regions to reduce the chances of a systemic collapse. Currency manipulation, foreign investment law, tarrifs, taxation, etc... are all lawful means of changing the playing field. In fact, the exact same techniques that countries use to protect against trade sanctions can be used to guard the economic safety of citizens locally. By playing by the current rules and free trade we are essentially playing into the hands of the larger companies of the world (mostly based in the United States). It's a form of imperialism/conquest (deliberate or not) without necessarily having to engage in open warfare and with the effective ruler being the United States with these companies acting as proxies

http://www.theglobeandmail.com/report-on-business/international-business/european-business/europe-shutters-as-greek-banks-bleed-cash/article25033867/

http://rt.com/business/250497-obama-economy-china-trade/

- making or saving money can sometimes be counterintuitive. If you've ever worked in the IT industry in any sort of support role then you'll realise that no matter what level of support you operate at one of the main aims is to establish whether or not the problem occurs without your own area of oversight. If it is, you try and fix it, if not you ignore it and basically tell the other end to kindly go away because you often don't have the expertise to fix it, nor do you have the oversight to be able to. The medical and pharmaceutical industry is much the same. The irony is that this perspective can result in longer term harm than good. The United States budget is out of whack with one of the major causes being the high cost of drugs as well as short sighted perspective of medical practitioners who tend to not attempt to treat the problem till it's fixed but keep on managing it. Fix it if you can and the problem goes away, your budget is in better shape

http://ourfiniteworld.com/2011/04/08/whats-behind-united-states-budget-problems/

http://www.businessinsider.com/us-budget-deficit-2011-7

- if so many countries are so concerned about profit shifting why don't they simply make it un-economical/impossible to re-locate from now on? That way existing financial centres for such activity can adapt in the meantime while others countries can begin to regain some of their investment

- every company engages in anti-competitive behaviour. Even though (and others) Google are a supposed proponent of the 'Don't be Evil' mantra they still have shareholders to report to meaning that even if they don't want to they have to

- if too few countries make changes their companies are going to be subject to foreign takeover interest (friendly and non-friendly) if adequate measures aren't taken to protect them. Moreover, they will be at a competitive disadvantage when attemping to branch out. The only way to look after these interests is to look at the way companies are structured in order to look after the needs of both the individual and collective simultaneously

- making changes for a fairer and more equatible society isn't easy and the irony is that those who are already successful will always appeal to reduce the chances of the status quo changing. They will insist that since they 'made it' so can others. Moreover, there are always those within the political and public services who will always have differing opinions on how to acheive the same thing

http://www.smh.com.au/world/us-supreme-court-hands-obama-major-victory-on-obamacare-healthcare-reform-20150625-ghy1xq.html#content

http://www.seattletimes.com/seattle-news/a-pathological-refusal-to-see-any-shred-of-good-in-obamacare/

- people say that globalisation and free market capitalism is a guard against collapse. Someone in the system is always going to be looking for money or someone is always going to have money. The problem is that there's no incentive to do this. Moreover, it has been proven in the United States and Europe that pure private, free trade capitalism isn't necessarily going to fill the void should there be significant underlying problems. Even states and unions can not hold back the dam should the market burst. Moreover, firms have shareholders and creditors to report to. Without adequate safeguards in place the needs of the many are never going to be met by the few who are lucky enough to have survived (there is only one exception to this. If there is strong leadership/management in the private sector which I haven't seen to many instances of)

https://en.wikipedia.org/wiki/Great_Recession

http://www.afr.com/markets/commodities/energy/saudis-seen-escalating-battle-for-global-oil-market-share-20150618-ghrxws

https://en.wikipedia.org/wiki/2007%E2%80%9308_world_food_price_crisis

https://en.wikipedia.org/wiki/2000s_energy_crisis

http://www.news.com.au/national/breaking-news/govt-to-explore-social-impact-bonds/story-e6frfku9-1227416495203

http://www.news.com.au/world/breaking-news/pope-talking-drivel-catholic-economist/story-e6frfkui-1227416020721

Donna Benjamin: Try Drupal with Blackmesh

Sat, 2015-06-27 00:27
Friday, June 26, 2015 - 23:33Try Drupal

Arjen Lentz: Dutch Court orders Netherlands Government cut CO2 emissions by 25 percent by 2020 | Climate Citizen

Thu, 2015-06-25 20:25

http://takvera.blogspot.com.au/2015/06/dutch-court-orders-netherlands.html

A Dutch court in a landmark legal case has just handed down a verdict that the Netherlands Government has the legal duty to take measures against #climate change. Further, the court ordered that a 25% reduction of CO2 emissions, based on 1990 levels, must be accomplished by 2020 by the Dutch government in accordance with IPCC scientific recommendations for industrial countries.

[…]

Sue Higginson, Principal Solicitor for the Environmental Defenders Office (EDO) NSW, said that the same legal arguments are unlikely to be used in Australia, “Dutch civil laws are much more specific in their terms than Australian laws.” she said.

[…]

With Australia, such a case would be much less straightforward as we do not have the incorporation of international human rights or general duty of care directly in our constitution or legal framework.

Rusty Russell: Hashing Speed: SHA256 vs Murmur3

Thu, 2015-06-25 18:29

So I did some IBLT research (as posted to bitcoin-dev ) and I lazily used SHA256 to create both the temporary 48-bit txids, and from them to create a 16-bit index offset.  Each node has to produce these for every bitcoin transaction ID it knows about (ie. its entire mempool), which is normally less than 10,000 transactions, but we’d better plan for 1M given the coming blopockalypse.

For txid48, we hash an 8 byte seed with the 32-byte txid; I ignored the 8 byte seed for the moment, and measured various implementations of SHA256 hashing 32 bytes on on my Intel Core i3-5010U CPU @ 2.10GHz laptop (though note we’d be hashing 8 extra bytes for IBLT): (implementation in CCAN)

  1. Bitcoin’s SHA256: 527.7+/-0.9 nsec
  2. Optimizing the block ending on bitcoin’s SHA256: 500.4+/-0.66 nsec
  3. Intel’s asm rorx: 314.1+/-0.3 nsec
  4. Intel’s asm SSE4 337.5+/-0.5 nsec
  5. Intel’s asm RORx-x8ms 458.6+/-2.2 nsec
  6. Intel’s asm AVX 336.1+/-0.3 nsec

So, if you have 1M transactions in your mempool, expect it to take about 0.62 seconds of hashing to calculate the IBLT.  This is too slow (though it’s fairly trivially parallelizable).  However, we just need a universal hash, not a cryptographic one, so I benchmarked murmur3_x64_128:

  1. Murmur3-128: 23 nsec

That’s more like 0.046 seconds of hashing, which seems like enough of a win to add a new hash to the mix.