Planet Linux Australia

Syndicate content
Planet Linux Australia - http://planet.linux.org.au
Updated: 1 hour 29 sec ago

sthbrx - a POWER technical blog: NCSI - Nice Network You've Got There

Fri, 2017-09-22 11:08

A neat piece of kernel code dropped into my lap recently, and as a way of processing having to inject an entire network stack into by brain in less-than-ideal time I thought we'd have a look at it here: NCSI!

NCSI - Not the TV Show

NCSI stands for Network Controller Sideband Interface, and put most simply it is a way for a management controller (eg. a BMC like those found on our OpenPOWER machines) to share a single physical network interface with a host machine. Instead of two distinct network interfaces you plug in a single cable and both the host and the BMC have network connectivity.

NCSI-capable network controllers achieve this by filtering network traffic as it arrives and determining if it is host- or BMC-bound. To know how to do this the BMC needs to tell the network controller what to look out for, and from a Linux driver perspective this the focus of the NCSI protocol.

Hi My Name Is 70:e2:84:14:24:a1

The major components of what NCSI helps facilitate are:

  • Network Controllers, known as 'Packages' in this context. There may be multiple separate packages which contain one or more Channels.
  • Channels, most easily thought of as the individual physical network interfaces. If a package is the network card, channels are the individual network jacks. (Somewhere a pedant's head is spinning in circles).
  • Management Controllers, or our BMC, with their own network interfaces. Hypothetically there can be multiple management controllers in a single NCSI system, but I've not come across such a setup yet.

NCSI is the medium and protocol via which these components communicate.

The interface between Management Controller and one or more Packages carries both general network traffic to/from the Management Controller as well as NCSI traffic between the Management Controller and the Packages & Channels. Management traffic is differentiated from regular traffic via the inclusion of a special NCSI tag inserted in the Ethernet frame header. These management commands are used to discover and configure the state of the NCSI packages and channels.

If a BMC's network interface is configured to use NCSI, as soon as the interface is brought up NCSI gets to work finding and configuring a usable channel. The NCSI driver at first glance is an intimidating combination of state machines and packet handlers, but with enough coffee it can be represented like this:

Without getting into the nitty gritty details the overall process for configuring a channel enough to get packets flowing is fairly straightforward:

  • Find available packages.
  • Find each package's available channels.
  • (At least in the Linux driver) select a channel with link.
  • Put this channel into the Initial Config State. The Initial Config State is where all the useful configuration occurs. Here we find out what the selected channel is capable of and its current configuration, and set it up to recognise the traffic we're interested in. The first and most basic way of doing this is configuring the channel to filter traffic based on our MAC address.
  • Enable the channel and let the packets flow.

At this point NCSI takes a back seat to normal network traffic, transmitting a "Get Link Status" packet at regular intervals to monitor the channel.

AEN Packets

Changes can occur from the package side too; the NCSI package communicates these back to the BMC with Asynchronous Event Notification (AEN) packets. As the name suggests these can occur at any time and the driver needs to catch and handle these. There are different types but they essentially boil down to changes in link state, telling the BMC the channel needs to be reconfigured, or to select a different channel. These are only transmitted once and no effort is made to recover lost AEN packets - another good reason for the NCSI driver to periodically monitor the channel.

Filtering

Each channel can be configured to filter traffic based on MAC address, broadcast traffic, multicast traffic, and VLAN tagging. Associated with each of these filters is a filter table which can hold a finite number of entries. In the case of the VLAN filter each channel could match against 15 different VLAN IDs for example, but in practice the physical device will likely support less. Indeed the popular BCM5718 controller supports only two!

This is where I dived into NCSI. The driver had a lot of the pieces for configuring VLAN filters but none of it was actually hooked up in the configure state, and didn't have a way of actually knowing which VLAN IDs were meant to be configured on the interface. The bulk of that work appears in this commit where we take advantage of some useful network stack callbacks to get the VLAN configuration and set them during the configuration state. Getting to the configuration state at some arbitrary time and then managing to assign multiple IDs was the trickiest bit, and is something I'll be looking at simplifying in the future.

NCSI! A neat way to give physically separate users access to a single network controller, and if it works right you won't notice it at all. I'll surely be spending more time here (fleshing out the driver's features, better error handling, and making the state machine a touch more readable to start, and I haven't even mentioned HWA), so watch this space!

OpenSTEM: Those Dirty Peasants!

Mon, 2017-09-18 10:04
It is fairly well known that many Europeans in the 17th, 18th and early 19th centuries did not follow the same routines of hygiene as we do today. There are anecdotal and historical accounts of people being dirty, smelly and generally unhealthy. This was particularly true of the poorer sections of society. The epithet “those […]

Dave Hall: Trying Drupal

Sat, 2017-09-16 22:03

While preparing for my DrupalCamp Belgium keynote presentation I looked at how easy it is to get started with various CMS platforms. For my talk I used Contentful, a hosted content as a service CMS platform and contrasted that to the "Try Drupal" experience. Below is the walk through of both.

Let's start with Contentful. I start off by visiting their website.

In the top right corner is a blue button encouraging me to "try for free". I hit the link and I'm presented with a sign up form. I can even use Google or GitHub for authentication if I want.

While my example site is being installed I am presented with an overview of what I can do once it is finished. It takes around 30 seconds for the site to be installed.

My site is installed and I'm given some guidance about what to do next. There is even an onboarding tour in the bottom right corner that is waving at me.

Overall this took around a minute and required very little thought. I never once found myself thinking come on hurry up.

Now let's see what it is like to try Drupal. I land on d.o. I see a big prominent "Try Drupal" button, so I click that.

I am presented with 3 options. I am not sure why I'm being presented options to "Build on Drupal 8 for Free" or to "Get Started Risk-Free", I just want to try Drupal, so I go with Pantheon.

Like with Contentful I'm asked to create an account. Again I have the option of using Google for the sign up or completing a form. This form has more fields than contentful.

I've created my account and I am expecting to be dropped into a demo Drupal site. Instead I am presented with a dashboard. The most prominent call to action is importing a site. I decide to create a new site.

I have to now think of a name for my site. This is already feeling like a lot of work just to try Drupal. If I was a busy manager I would have probably given up by this point.

When I submit the form I must surely be going to see a Drupal site. No, sorry. I am given the choice of installing WordPress, yes WordPress, Drupal 8 or Drupal 7. Despite being very confused I go with Drupal 8.

Now my site is deploying. While this happens there is a bunch of items that update above the progress bar. They're all a bit nerdy, but at least I know something is happening. Why is my only option to visit my dashboard again? I want to try Drupal.

I land on the dashboard. Now I'm really confused. This all looks pretty geeky. I want to try Drupal not deal with code, connection modes and the like. If I stick around I might eventually click "Visit Development site", which doesn't really feel like trying Drupal.

Now I'm asked to select a language. OK so Drupal supports multiple languages, that nice. Let's select English so I can finally get to try Drupal.

Next I need to chose an installation profile. What is an installation profile? Which one is best for me?

Now I need to create an account. About 10 minutes I already created an account. Why do I need to create another one? I also named my site earlier in the process.

Finally I am dropped into a Drupal 8 site. There is nothing to guide me on what to do next.

I am left with a sense that setting up Contentful is super easy and Drupal is a lot of work. For most people wanting to try Drupal they would have abandoned someway through the process. I would love to see the conversion stats for the try Drupal service. It must miniscule.

It is worth noting that Pantheon has the best user experience of the 3 companies. The process with 1&1 just dumps me at a hosting sign up page. How does that let me try Drupal?

Acquia drops onto a page where you select your role, then you're presented with some marketing stuff and a form to request a demo. That is unless you're running an ad blocker, then when you select your role you get an Ajax error.

The Try Drupal program generates revenue for the Drupal Association. This money helps fund development of the project. I'm well aware that the DA needs money. At the same time I wonder if it is worth it. For many people this is the first experience they have using Drupal.

The previous attempt to have simplytest.me added to the try Drupal page ultimately failed due to the financial implications. While this is disappointing I don't think simplytest.me is necessarily the answer either.

There needs to be some minimum standards for the Try Drupal page. One of the key item is the number of clicks to get from d.o to a working demo site. Without this the "Try Drupal" page will drive people away from the project, which isn't the intention.

If you're at DrupalCon Vienna and want to discuss this and other ways to improve the marketing of Drupal, please attend the marketing sprints.

AttachmentSize try-contentful-1.png342.82 KB try-contentful-2.png214.5 KB try-contentful-3.png583.02 KB try-contentful-5.png826.13 KB try-drupal-1.png1.19 MB try-drupal-2.png455.11 KB try-drupal-3.png330.45 KB try-drupal-4.png239.5 KB try-drupal-5.png203.46 KB try-drupal-6.png332.93 KB try-drupal-7.png196.75 KB try-drupal-8.png333.46 KB try-drupal-9.png1.74 MB try-drupal-10.png1.77 MB try-drupal-11.png1.12 MB try-drupal-12.png1.1 MB try-drupal-13.png216.49 KB

OpenSTEM: New Dates for Human Relative + ‘Explorer Classroom’ Resources

Thu, 2017-09-14 12:04
During September, National Geographic is featuring the excavations of Homo naledi at Rising Star Cave in South Africa in their Explorer Classroom, in tune with new discoveries and the publishing of dates for this enigmatic little hominid. A Teacher’s Guide and Resources are available and classes can log in to see live updates from the […]

OpenSTEM: Guess the Artefact #3

Mon, 2017-09-11 10:04
This week’s Guess the Artefact challenge centres around an artefact used by generations of school children. There are some adults who may even have used these themselves when they were at school. It is interesting to see if modern students can recognise this object and work out how it was used. The picture below comes […]

Russell Coker: Observing Reliability

Sat, 2017-09-09 22:02

Last year I wrote about how great my latest Thinkpad is [1] in response to a discussion about whether a Thinkpad is still the “Rolls Royce” of laptops.

It was a few months after writing that post that I realised that I omitted an important point. After I had that laptop for about a year the DVD drive broke and made annoying clicking sounds all the time in addition to not working. I removed the DVD drive and the result was that the laptop was lighter and used less power without missing any feature that I desired. As I had installed Debian on that laptop by copying the hard drive from my previous laptop I had never used the DVD drive for any purpose. After a while I got used to my laptop being like that and the gaping hole in the side of the laptop where the DVD drive used to be didn’t even register to me. I would prefer it if Lenovo sold Thinkpads in the T series without DVD drives, but it seems that only the laptops with tiny screens are designed to lack DVD drives.

For my use of laptops this doesn’t change the conclusion of my previous post. Now the T420 has been in service for almost 4 years which makes the cost of ownership about $75 per year. $1.50 per week as a tax deductible business expense is very cheap for such a nice laptop. About a year ago I installed a SSD in that laptop, it cost me about $250 from memory and made it significantly faster while also reducing heat problems. The depreciation on the SSD about doubles the cost of ownership of the laptop, but it’s still cheaper than a mobile phone and thus not in the category of things that are expected to last for a long time – while also giving longer service than phones usually do.

One thing that’s interesting to consider is the fact that I forgot about the broken DVD drive when writing about this. I guess every review has an unspoken caveat of “this works well for me but might suck badly for your use case”. But I wonder how many other things that are noteworthy I’m forgetting to put in reviews because they just don’t impact my use. I don’t think that I am unusual in this regard, so reading multiple reviews is the sensible thing to do.

Related posts:

  1. Is a Thinkpad Still Like a Rolls-Royce For a long time the Thinkpad has been widely regarded...
  2. PC prices drop again! A few weeks ago Dell advertised new laptops for $849AU,...
  3. Laptop Reliability Update: TumbleDry has a good analysis of the Square Trade...

Francois Marier: TLS Authentication on Freenode and OFTC

Sat, 2017-09-09 15:52

In order to easily authenticate with IRC networks such as OFTC and Freenode, it is possible to use client TLS certificates (also known as SSL certificates). In fact, it turns out that it's very easy to setup both on irssi and on znc.

Generate your TLS certificate

On a machine with good entropy, run the following command to create a keypair that will last for 10 years:

openssl req -nodes -newkey rsa:2048 -keyout user.pem -x509 -days 3650 -out user.pem -subj "/CN=<your nick>"

Then extract your key fingerprint using this command:

openssl x509 -sha1 -noout -fingerprint -in user.pem | sed -e 's/^.*=//;s/://g' Share your fingerprints with NickServ

On each IRC network, do this:

/msg NickServ IDENTIFY Password1! /msg NickServ CERT ADD <your fingerprint>

in order to add your fingerprint to the access control list.

Configure ZNC

To configure znc, start by putting the key in the right place:

cp user.pem ~/.znc/users/<your nick>/networks/oftc/moddata/cert/

and then enable the built-in cert plugin for each network in ~/.znc/configs/znc.conf:

<Network oftc> ... LoadModule = cert ... </Network> <Network freenode> ... LoadModule = cert ... </Network> Configure irssi

For irssi, do the same thing but put the cert in ~/.irssi/user.pem and then change the OFTC entry in ~/.irssi/config to look like this:

{ address = "irc.oftc.net"; chatnet = "OFTC"; port = "6697"; use_tls = "yes"; tls_cert = "~/.irssi/user.pem"; tls_verify = "yes"; autoconnect = "yes"; }

and the Freenode one to look like this:

{ address = "chat.freenode.net"; chatnet = "Freenode"; port = "7000"; use_tls = "yes"; tls_cert = "~/.irssi/user.pem"; tls_verify = "yes"; autoconnect = "yes"; }

That's it. That's all you need to replace password authentication with a much stronger alternative.

James Morris: Linux Plumbers Conference Sessions for Linux Security Summit Attendees

Thu, 2017-09-07 14:01

Folks attending the 2017 Linux Security Summit (LSS) next week may be also interested in attending the TPMs and Containers sessions at Linux Plumbers Conference (LPC) on the Wednesday.

The LPC TPMs microconf will be held in the morning and lead by Matthew Garret, while the containers microconf will be run by Stéphane Graber in the afternoon.  Several security topics will be discussed in the containers session, including namespacing and stacking of LSM, and namespacing of IMA.

Attendance on the Wednesday for LPC is at no extra cost for registered attendees of LSS.  Many thanks to the LPC organizers for arranging this!

There will be followup BOF sessions on LSM stacking and namespacing at LSS on Thursday, per the schedule.

This should be a very productive week for Linux security development: see you there!

Linux Users of Victoria (LUV) Announce: Software Freedom Day 2017 and LUV Annual General Meeting

Wed, 2017-09-06 14:03
Start: Sep 16 2017 12:00 End: Sep 16 2017 18:00 Start: Sep 16 2017 12:00 End: Sep 16 2017 18:00 Location:  Electron Workshop, 31 Arden St. North Melbourne Link:  http://www.electronworkshop.com.au/map/  Software Freedom Day 2017

It's that time of the year where we celebrate our freedoms in technology and raise a ruckus about all the freedoms that have been eroded away. The time of the year we look at how we might keep our increasingly digital lives under our our own control and prevent prying eyes from seeing things they shouldn't. You guessed it: It's Software Freedom Day!

LUV would like to acknowledge Electron Workshop for the venue.

Linux Users of Victoria Inc., is an incorporated association, registration number A0040056C.

September 16, 2017 - 12:00

read more

OpenSTEM: Space station trio returns to Earth: NASA’s Peggy Whitson racks up 665-day record | GeekWire

Wed, 2017-09-06 10:04
https://www.geekwire.com/2017/space-station-trio-returns-earth-nasas-peggy-whitson-sets-665-day-record/ NASA astronaut Peggy Whitson and two other spacefliers capped off a record-setting orbital mission with their return from the International Space Station.

Tim Serong: Understanding BlueStore, Ceph’s New Storage Backend

Mon, 2017-09-04 20:05

On June 1, 2017 I presented Understanding BlueStore, Ceph’s New Storage Backend at OpenStack Australia Day Melbourne. As the video is up (and Luminous is out!), I thought I’d take the opportunity to share it, and write up the questions I was asked at the end.

First, here’s the video:

The bit at the start where the audio cut out was me asking “Who’s familiar with Ceph?” At this point, most of the 70-odd people in the room put their hands up. I continued with “OK, so for the two people who aren’t…” then went into the introduction.

After the talk we had a Q&A session, which I’ve paraphrased and generally cleaned up here.

With BlueStore, can you still easily look at the objects like you can through the filesystem when you’re using FileStore?

There’s not a regular filesystem anymore, so you can’t just browse through it. However you can use `ceph-objectstore-tool` to “mount” an offline OSD’s data via FUSE and poke around that way. Some more information about this can be found in Sage Weil’s recent blog post: New in Luminous: BlueStore.

Do you have real life experience with BlueStore for how IOPS performance scales?

We (SUSE) haven’t released performance numbers yet, so I will instead refer you to Sage Weil’s slides from Vault 2017, and Allan Samuel’s slides from SCALE 15x, which together include a variety of performance graphs for different IO patterns and sizes. Also, you can expect to see more about this on the Ceph blog in the coming weeks.

What kind of stress testing has been done for corruption in BlueStore?

It’s well understood by everybody that it’s sort of important to stress test these things and that people really do care if their data goes away. Ceph has a huge battery of integration tests, various of which are run on a regular basis in the upstream labs against Ceph’s master and stable branches, others of which are run less frequently as needed. The various downstreams all also run independent testing and QA.

Wouldn’t it have made sense to try to enhance existing POSIX filesystems such as XFS, to make them do what Ceph needs?

Long answer: POSIX filesystems still need to provide POSIX semantics. Changing the way things work (or adding extensions to do what Ceph needs) in, say, XFS, assuming it’s possible at all, would be a big, long, scary, probably painful project.

Short answer: it’s really a different use case; better to build a storage engine that fits the use case, than shoehorn in one that doesn’t.

Best answer: go read New in Luminous: BlueStore ;-)

OpenSTEM: This Week in HASS – term 3, week 9

Mon, 2017-09-04 10:04
OpenSTEM’s ® Understanding Our World® Units are designed to cover 9 weeks of the term, because we understand that life happens. Sports carnivals, excursions and other special events are also necessary parts of the school year and even if the calendar runs according to plan, having a little bit of breathing space at the end of […]

sthbrx - a POWER technical blog: memcmp() for POWER8 - part II

Fri, 2017-09-01 13:00

This entry is a followup to part I which you should absolutely read here before continuing on.

Where we left off

We concluded that while a vectorised memcmp() is a win, there are some cases where it won't quite perform.

The overhead of enabling ALTIVEC

In the kernel we explicitly don't touch ALTIVEC unless we need to, this means that in the general case we can leave the userspace registers in place and not have do anything to service a syscall for a process.

This means that if we do want to use ALTIVEC in the kernel, there is some setup that must be done. Notably, we must enable the facility (a potentially time consuming move to MSR), save off the registers (if userspace we using them) and an inevitable restore later on.

If all this needs to be done for a memcmp() in the order of tens of bytes then it really wasn't worth it.

There are two reasons that memcmp() might go for a small number of bytes, firstly and trivially detectable is simply that parameter n is small. The other is harder to detect, if the memcmp() is going to fail (return non zero) early then it also wasn't worth enabling ALTIVEC.

Detecting early failures

Right at the start of memcmp(), before enabling ALTIVEC, the first 64 bytes are checked using general purpose registers. Why the first 64 bytes, well why not? In a strange twist of fate 64 bytes happens to be the amount of bytes in four ALTIVEC registers (128 bits per register, so 16 bytes multiplied by 4) and by utter coincidence that happens to be the stride of the ALTIVEC compare loop.

What does this all look like

Well unlike part I the results appear slightly less consistent across three runs of measurement but there are some very key differences with part I. The trends do appear to be the same across all three runs, just less pronounced - why this is is unclear.

The difference between run two and run three clipped at deltas of 1000ns is interesting:

vs

The results are similar except for a spike in the amount of deltas in the unpatched kernel at around 600ns. This is not present in the first sample (deltas1) of data. There are a number of reasons why this spike could have appeared here, it is possible that the kernel or hardware did something under the hood, prefetch could have brought deltas for a memcmp() that would otherwise have yielded a greater delta into the 600ns range.

What these two graphs do both demonstrate quite clearly is that optimisations down at the sub 100ns end have resulted in more sub 100ns deltas for the patched kernel, a significant win over the original data. Zooming out and looking at a graph which includes deltas up to 5000ns shows that the sub 100ns delta optimisations haven't noticeably slowed the performance of long duration memcmp(), .

Conclusion

The small amount of extra development effort has yielded tangible results in reducing the low end memcmp() times. This second round of data collection and performance analysis only confirms the that for any significant amount of comparison, a vectorised loop is significantly quicker.

The results obtained here show no downside to adopting this approach for all power8 and onwards chips as this new version of the patch solves the performance regression for small compares.

Lev Lafayette: An Overview of SSH

Thu, 2017-08-31 00:04

SSH (Secure Shell) is secure means, mainly used on Linux and other UNIX-like systems, to access remote systems, designed as a replacement for a variety of insecure protocols (e.g., telnet, rlogin etc). This presentation will cover the core useage of the protocol, its development (SSH-1 and SSH-2), the architecture (client-server, public key authentication), installation and implementation, and some handy elaborations and enhancements, and real and imagined vulnerabilities.
Plenty of examples will be provided throughout along with the opportunity to test the protocol.

read more

Linux Users of Victoria (LUV) Announce: LUV Main September 2017 Meeting: Cygwin and Virtualbox

Tue, 2017-08-29 18:03
Start: Sep 5 2017 18:30 End: Sep 5 2017 20:30 Start: Sep 5 2017 18:30 End: Sep 5 2017 20:30 Location:  The Dan O'Connell Hotel, 225 Canning Street, Carlton VIC 3053 Link:  http://luv.asn.au/meetings/map

Tuesday, September 5, 2017

6:30 PM to 8:30 PM
The Dan O'Connell Hotel
225 Canning Street, Carlton VIC 3053

Speakers:

  • Duncan Roe, Cygwin
  • Steve Roylance, Virtualbox

Cygwin is a large collection of GNU and Open Source tools which provide functionality similar to a Linux distribution on Windows.  It allows easy porting of many Unix programs without the need for extensive changes to the source code.

The Dan O'Connell Hotel, 225 Canning Street, Carlton VIC 3053

Food and drinks will be available on premises.

Before and/or after each meeting those who are interested are welcome to join other members for dinner.

Linux Users of Victoria Inc., is an incorporated association, registration number A0040056C.

September 5, 2017 - 18:30

read more

BlueHackers: Creative kid’s piano + vocal composition

Mon, 2017-08-28 11:01

An inspirational song from an Australian youngster.  He explains his background at the start.

OpenSTEM: This Week in HASS – term 3, week 8

Mon, 2017-08-28 10:05
This week our younger students are putting together a Class Museum, while older students are completing their Scientific Report. Foundation/Prep/Kindy to Year 3 Students in Foundation/Prep/Kindy (Units F.3 and F-1.3), as well as those in Years 1 (Unit 1.3). 2 (Unit 2.3) and 3 (Unit 3.3) are all putting together Class Museums of items of […]