Cody Bunch Some Random IT Guy - OpenStack, DevOps, Cloud, Things

Basic Server Hardening with Salt

Before we get started, take a gander at my last few posts on this, here and here.

The idea here is roughly the same. That is, build a small, basic ‘base’ profile, template, state, or whatever, that has some simple hardening bits applied. The idea being to give you a reasonable start in turn letting you apply additional layers down the road.

The reason you’d move away from doing this with OpenStack Orchestration (Heat) and into a config management tool is that it allows you to apply the same practices more generically. That is not everyone runs OpenStack, but lots of folks are moving to some flavor of configuration management tool, if they weren’t there already. You can then include these SaltStates in Heat or whatever orchestration tool of choice.

Basic Hardening with SaltStack

For the sake of not making this a huge-tastic blog post, we’ll skip the part where I explain the what and how of getting started with SaltStack. Many others have done better than I. What follows is my top.sls, secureserver.sls and sysctl.sys files.

top.sls

This file controls what Salt minions get what ‘states’. For securing all the servers, that’s pretty straight forward. I would like to call out, however, the ordering of the states:

base:
  '*':
    - sysctl
    - secureserver

This defines a ‘base’ state that will match for all Salt minions. (Minion is Salt for agent.) It then states to apply first the sysctl state, then the secureserver state.

sysctl.sls

There isn’t much fancy to this file. In fact, it’s more or less a YAML formatted version of what you’d throw into /etc/sysctl.conf. What the lines below do is turn off IP routing, ignore broadcasts, responses, and all manner of other fun icmp stuff.

net.ipv4.icmp_echo_ignore_broadcasts:
  sysctl.present:
    - value: 1

net.ipv4.icmp_ignore_bogus_error_responses:
  sysctl.present:
  - value: 1

net.ipv4.tcp_syncookies:
  sysctl.present:
    - value: 1

net.ipv4.conf.all.log_martians:
  sysctl.present:
  - value: 1

net.ipv4.conf.default.log_martians:
  sysctl.present:
    - value: 1

net.ipv4.conf.all.accept_source_route:
  sysctl.present:
    - value: 0

net.ipv4.conf.default.accept_source_route:
  sysctl.present:
    - value: 0

net.ipv4.conf.all.rp_filter:
  sysctl.present:
    - value: 1

net.ipv4.conf.default.rp_filter:
  sysctl.present:
    - value: 1

net.ipv4.conf.all.accept_redirects:
  sysctl.present:
    - value: 0

net.ipv4.conf.default.accept_redirects:
  sysctl.present:
    - value: 0

net.ipv4.conf.all.secure_redirects:
  sysctl.present:
    - value: 0

net.ipv4.conf.default.secure_redirects:
  sysctl.present:
    - value: 0

net.ipv4.ip_forward:
  sysctl.present:
    - value: 0

net.ipv4.conf.all.send_redirects:
    sysctl.present:
    - value: 0

net.ipv4.conf.default.send_redirects:
  sysctl.present:
    - value: 0

secureserver.sls

This file, unlike the sysctl bits, actually does some install and config bits. If you’ve read the post I mentioned at the beginning of this post, you’ll recognize the packages being installed.

The configure bits are two fold: 1) We configure logwatch using a file hosted on the Salt master; 2) We configure IP tables to allow ssh and deny all the other things:

iptables:
  pkg:
    - installed

denyhosts:
  pkg:
    - installed

psad:
  pkg:
    - installed

fail2ban:
  pkg:
    - installed

logwatch:
  pkg:
    - installed

aide:
  pkg:
    - installed

/etc/cron.daily/00logwatch:
  file:
    - managed
    - source: salt://cron.daily/00logwatch
    - require:
      - pkg: logwatch

ssh:
  iptables.append:
    - table: filter
    - chain: INPUT
    - jump: ACCEPT
    - match: state
    - connstate: NEW
    - dport: 22
    - proto: tcp
    - sport: 1025:65535
    - save: True

allow established:
  iptables.append:
    - table: filter
    - chain: INPUT
    - match: state
    - connstate: ESTABLISHED
    - jump: ACCEPT

default to reject:
  iptables.append:
    - table: filter
    - chain: INPUT
    - jump: REJECT

Summary

In this post we’ve covered how to do some very very basic hardening of your server using SaltStack. This likely wont work for all circumstances (like if you’re going to actually run nginx or apache, you need to add those ports accordingly).

Resources

Link Dump - Random Edition

Today’s link dump is brought to you by Google Chrome and my poor swap file.

On OpenStack 2015 Board Elections

Sometime back in November, I received an email stating that I had been nominated, by the OpenStack community, to run for an Individual Board Member position. It was very shortly thereafter I had the 10 needed nominations to get on the ballot. I was super excited at the prospect, and am super humbled that I’d even be considered.

Let me say that again. I am incredibly humbled that the community reached out and hopped on to support my nomination.

I repeat that, because at this time, I’m deciding to back out of the election for two reasons. First and foremost, family considerations. Due to unforeseen family circumstances, I need to take a few steps back from the various things I am involved in for a while.

My second reason for backing out, is the entry of Egle (@eglute AnyStacker) into the elections. Having worked closely with Egle on a number of workshops, books, and work projects over the last few years, I can say that y’all will be in great hands if she’s elected.

Thank you again for your all your support. Mayhaps next time.

Unbreak Email in 2015: 3 folders, 2 times a day, 1 rule

With the new year upon us, a lot of you will likely make some manner of commitment to be better at handling communications, process email, and in general, get things done.

Here’s a system I’ve built / adapted from others who are much more effective at email than I am. The system in general has helped reduce stress and help me focus on and engage better with the parts of email that matter. Because I am lazy, it’s also super simple and automated to a degree. I am not prescribing this as a fix to all of your email woes, rather, suggesting that like me, you read, learn, and adapt it to help improve how you handle email next year. (ZOMG RUNON)

It’s got some basic components, and because lists are a good SEO / Click-Bait thing, that’s how we’ll arrange it:

Unbreak Email

  • 3 Folders
  • 2 Times a day
  • 1 Processing Rule

3 Folders

It’s actually two for processing and one for storing reference material. These folders are:

-> Inbox
|-> I'm Awesome
|-> Done

The basic workflow is that everything lands in the inbox, and gets processed into either the “I’m Awesome” (or Kudos, etc) folder or into the Done folder.

What is this “I’m Awesome” folder? It actually serves a few needs.

First and foremost it’s a tool to be used around review time. That is, you take any email where someone thanks you for a job well done, a contribution to a project, and other similar things, and place them here. If you do self-reviews, retrospectives, or other similar management things, it is handy to have this as a reminder of the contributions you’ve made over that time period, and if need be, remind The Man™.

Second, and no less important, is this folders ability to recharge your batteries. If you start to experience burn out, feel like you aren’t having an impact, and other similar feelings, looking back at this folder should remind you a bit about why you do what you do.

2 Times A Day

I generally check email twice a day. That is, across all accounts (gmail, provmware, work, etc). Twice a day.

This isn’t a hard and fast rule. That is, emergencies and other high priority things happen in life that necessitate checking with more frequency.

For things like collaboration, team communication, social, and more, there are more and better, near instant forms of communication.

During these processing times, process the “Inbox” folder first, as this contains everything that is addressed to me, or needs input. Giving these priority let’s you address the 20% of email that needs 80% of your attention.

Next, process the “Done” folder which will contain mostly automated emails and email list mails. In this case, the “One Rule” discussed next puts anything and everything that is not addressed directly to you into the Done folder. This is because in the majority of circumstances, mail coming to an email list or from an automated source is informative, but not of major consequence if missed or filed as “Done”.

As these emails in this folder are largely informative in nature, skimming them has worked well for me. Skimming strategies however, are best left for another post (or some manner of productivity expert).

1 Email Rule

The 1 rule to rule them all that I use to support all of the above:

“On incoming mail, where I am not in the TO: or CC: field, move to ‘Done’”.

This enables the bits above, and in one fell swoop, will reduce your load significantly.

Holiday Zen Moments

Apologies in advance if this is a bit more personal than technical. There is plenty more tech content coming, have no fear.

I ride because I ride

It’s the holidays WOOOOO! Well, maybe no seven O’s woo, but still, a good time nevertheless. On the Zen moments thing, about 6 years ago, my father told me this story, and designed the sticker you see above.

The story:

Five students of a Zen master was back from the market on their bicycles. As they dismounted, their master asked: “Why are you riding your bicycles?”

Each of them came up with different answers to their master’s query.

The first student said: “It is the bicycle that is carrying the sack of potatoes. I am glad that my back has escaped the pain of bearing the weight.”

The master was glad and said: ”You are a smart boy. When you become old you will be saved of a hunch back unlike me.”

The second student had a different answer: ”I love to have my eyes over the trees and the sprawling fields as I go riding.”

The teacher commended: “You have your eyes open and you see the world.”

The third disciple came up with yet a different answer: ”When I ride I am content to chant ‘nam myoho renge kyo’”

The master spoke words of appreciation: ”Your mind will roll with ease like a newly trued wheel.”

The fourth disciple said: “Riding my bicycle I live in perfect harmony of things.”

The pleased master said: ”You are actually riding the golden path of non-harming or non violence.”

The fifth student said: ”I ride my bicycle to ride my bicycle.”

The master walked up to him and sat at his feet and said: “I am your disciple!”

Having ridden a bicycle for a number of years, I have used it for various means and in various phases. Weight-loss, transportation, racing, harmony with nature, etc. However, over the last several years through varied events, dramas and the like, I have learned that in cycling: “I ride because I ride”.

During the holidays, one can get caught up in the presents, people, dramas, and the ever present exhaustion of well, the holidays. Over the years, I’ve been in all of the above situations and then some. This year, like in cycling, I am trying to “Holidays because I Holidays”.

Regardless of how, what, or why you get together this season, try to take a moment, sit back and enjoy them as much as you can.

If you also ride because you ride, and would like a sticker, either email me (bunchc at gmail) or ping me on twitter and we can arrange something.

Live Blog - Keystone to Keystone Federation

Session details here.

Speakers:

  • Marek Denis - Research Fellow, CERN
  • Steve Martinelli - Software Developer, IBM
  • Joe Savak Sr. - Product Manager, Rackspace
  • Brad Topol - Distinguished Engineer, IBM

In this presentation, we describe the federated identity enhancements we have added to support Keystone to Keystone federation for enabling hybrid cloud functionality. We begin with an overview of key hybrid cloud use cases that have been identified by our stakeholders including those being encountered by OpenStack superuser CERN. We then discuss our SAML based approach for enabling Keystones to trust each other and provide authorization and role support for resources in hybrid cloud environments.

Live Blog

Lots of different folks interested in identity federtion, Academia, companies, lots and lots of folks.

Use cases? - Easy to confiture, cloud bursting, central policy point, federating out, federating in. Keep the client small. No new protocols.

“Federate In” - You already have identity provider, SAML, etc Folks already have SSO / Identity. Federate allows for use of existing credentials to work with OpenStack MSP’s.

“Federate Out” - That is, you setup a trust between on prem and off prem clouds.

Cern’s Use-Case

Cern has 70,000 cores, they need more to process ALL the data they produce. This requires federation out allows folks to use pay-as-you-go to hire out additional resources as needed.

Cern also needs to be able to allow folks to federate in from others in science community.

Now an interlude for Keystone classic Auth.

Federated identity in Incehouse - Integrate existing tools, SAML, etc. There is a diagram, it has lots of arrows, the gist is you send SAML to keystone, keystone gives you a token, and things are good. This worked, but not as well as it could. Mapping engine, that is, groups in one system are not the same as groups in others. Woo Mapping: “IBM Regular Employees” –> “regular_canada” etc.

New diagram for Federation in Juno. A lot more arrows. This time around, Keystone is the provider, and will provide some level of attestation to the other Keystone in the trust relationship. Once the trust is in place, the user passes the token to either.

The SAML generator takes the token and goes backwards. Token –> SAML Generator –> SAML Assertion

Now we’re at a slide covering all manner of config data. Important bits: Mapping is still a thing. You also need to ‘prime’ the SAML assertion pump.

keystone-manage now has a metadata generation thing.

Back to Cern: - 2 datacenters, OpenStack Cells, Cells not popular. 40k users in AD, and 12k more ADFS (Federation). Cern uses SAML2, and will be the first OpenStack in the world to allow federate-in to allow external entities to consume their resources.

Patches to the OpenStack and Keystone clients.

Looking forward:

  • Auth-N
  • Horizon Integration
  • More & Better Mapping
  • Fine Grained ACLs
  • More protocols

Live Blog - Cloud Security: Do you know where your workloads are running?

Session background can be found here.

Speaker: Raghu Yeluri, Intel Corporation

As an Enterprise and/or a Cloud service provider, you would have to ensure that all regulatory requirements for workload and data sovereignty are met. You have to answer the questions from your customers like:

where is my workload running? , Are my workloads running in a compliant location? , How can I trust the Integrity of the host servers on which my workloads are running , can you prove to me that my workloads and data have not violated policies? , How can I control via policy where my workload can and cannot migrate and run .

Live Blog

Geo Tag / Asset tags are set in a write once area. Can be almost anything, user provided names, gps coordinates, actual asset tags, and importantly, the certificate of attestation from the TPM.

Today we’re using Glance Image Registry to set the launch properties / policies: e.g. Only runs in France. The “Trust and Launch” scheduler filter runs last against the list of servers left. It then runs a variant of Open Attestation to say, “Which are trusted?”. From there, the scheduler will deploy. This is all automatic. Only setting the tags is manual. This same attestation happens during migrations as well.

This is how we enable boundary control. We added Horizon plugins to support launch policies. Extended Nova scheduler for location filtering, and glance for policies. Finally we provide a number of tools that work in conjunction (OAT, etc). The end-to-end bits can be done entirely in OSS however.

This should be upstream in Kilo, however, we (Intel) provide scripts to make this work in Icehouse & Juno.

Live Demo - Lol WiFi!

There is no maximum number of tags that can be set, things like GEO, PCI-DSS, etc can be set, these are then used to select the servers from there.

Magic number of policies: 5. This was in conversation with NIST and MSP’s

Looking forward:

  • Extend geo-tagging for volumes
  • Tenant-controlled encryption / decryption under controlled circumstances

Extend geo-tagging for volumes - Basically the above but for Cinder. The scheduler is pluggable, so we should be able to make this happen. The assumption being x86 storage host. This will be more difficult with traditional SAN/NAS due to not being TXT enabled… yet.

Paris Summit - Day 2

Day number 2, Day number 2! He touched the cloud!

#vBrownBag Day 2

Day number 2 had a number of awesome #vBrownBag tech talks, one can find that playlist here.

ZeroVM Sessions

We had a mini track going yesterday afternoon starting at 3:40 with Carina C. Zona kicking it off with “Good, Fast, Cheap: Pick 3”. The 4:40 session right on the heels was a 90minute hands on workshop with ZeroCloud, or that sweet spot where ZeroVM meets OpenStack Swift. The recordings / slides for these are not yet available however. If you would like to try your hands at the workshop, you should be able to with the info from here.

Other notes from Day 2

  • ZOMG The food in Paris.
  • ZOMG The deserts in Paris.
  • If the waiter tells you no, he means it.
  • Sausage that says «AAAAA» is not your friend.

OpenStack Cookbook - Now on Juno

With the summit this week, I feel it’s appropriate to announce that the scripts supporting the OpenStack Cookbook have been updated (and are mostly functional) on OpenStack Juno.

To get started with them, you’ll need either VMware Workstation/Fusion or Virtualbox, Vagrant, and Git.

git clone https://github.com/openstackcookbook/openstackcookbook -b juno
cd openstackcookbook
vagrant up

These scripts will be updated and expanded once again as we being writing on the third edition of the OpenStack Cookbook.

Paris Summit - Day 1

Hooray Paris!

Well, maybe. Having only seen the inside of the conference center and hotel lobby of the hotel nextdoor, I didn’t get to do much summiting today. Rather, the day was spent prepping for tomorrows workshop.

I do hear that the #vBrownBags are doing extremely well with 19 individual sessions recorded (and streamed?) today, and a full schedule of 100-ish more this week.