Kanban and Blocked Work

Whats the best way to handle blocked work in a Kanban system?

  1. Do you increase the WIP limit, thus freeing up time for team members to work on something else?
  2. Do you keep the WIP limit as-is, and raise awareness by having people sit idle while the root cause of the blockage is pursued?

My thought is the proper long-term approach is probably #2, but #1 seems a lot easier to implement. (Image credit)

kanban.png

JMESPATH and Visual Studio Code

The default format in which AWS CLI data is returned is JSON.  The JSON data that is returned can be queried with the query language JMESPath.

For example, using the AWS CLI, I can list the details of the volumes associated with my CLI profile.  (aws ec2 describe-volumes)

Example output:

{
    “Volumes”: [{
        “AvailabilityZone”: “myZone”,
        “VolumeType”: “standard”,
        “VolumeId”: “myVolumeID1”,
        “State”: “in-use”,
        “SnapshotId”: “mySnapID”,
        “CreateTime”: “2017-01-02T00:55:03.000Z”,
        “Size”: 1
},
{
        “AvailabilityZone”: “myZone”,
        “VolumeType”: “standard”,
        “VolumeId”: “myVolumeID2”,
        “State”: “in-use”,
        “SnapshotId”: “mySnapID”,
        “CreateTime”: “2016-01-02T00:55:03.000Z”,
        “Size”: 1
    }]
}

Now, lets say there are many more volumes returned from this CLI command and from that output I want to find the volume(s) that were created before a specific date. (January 01, 2017)

I can utilize the query option on the AWS CLI command to return specifics from the JSON results.  (aws ec2 describe-volumes –query ‘Volumes[?CreateTime<`2017-01-01`].VolumeId’)

However, sometimes getting the JMESPath syntax correct is not always easily.  The extra tools that I utilize to help me are Visual Studio Code with the JMESPath plugin.

JMESPath

So with this VSCode plugin, you can quickly validate your JMESPath syntax if you need.  One note when using the “–query” option on the CLI is that the processing is happening on the Client machine, unlike the “–filter” option that happens on the server.

vscode.png

AWS Lambda and S3

Below are a couple of problems I ran into when writing a Python 2.7 Lambda function that created a file and then uploaded it to S3. (s3.upload_file)

  1. The file I was creating and writing to in the function was empty in S3 after the upload.
    • Turns out I needed the “( )” braces on the Python “close” command.  Silly issue, but took my like 20 minutes to figure out….
  2. In your Lambda function, you need to create your files under /tmp, which is your functions ephemeral storage.
    • fileName = ‘/tmp/’ + name

s3_image

Fabric and Reoccurring Tasks

So you have a handful of commands you execute weekly on multiple Linux servers.  You run these commands by opening a SSH tool like Putty, logging into the server, and then executing the commands.

Example commands:

  • ls /tmp/dropfolder
  • tail -10 /tmp/dropfolder/log1

With Fabric, you can bundle these commands and many others in a Fabric command file call fabfile.py.  Below is an example fabfile.py file containing just 1 task that will run (fabric.operations.run) the commands above.

from fabric.api import run

def dailycommands():
     run ("ls /tmp/dropfolder")
     run("tail -10 /tmp/dropfolder/log1")

You can then execute this file via the command line on 1 or multiple hosts with ease.

Example command:

  • fab -f fabfile.py -H host1,host2,host3 dailycommands

There are many many other things you can do with Fabric, so give it a try!

logo.png

Katalon – Docker – Linux

I’ve used Katalon primary in a Windows environment for running smoke tests.  This tool is relatively new, built on top of Selenium, and actively supported.  I highly suggest checking it out if you have some free time.  (Article describe Katalon pro’s)

So, the Katalon tests I run are launched from Jenkins and executed on a Windows 2012 slave machines via the command line.  These slave machines must have Katalon installed in order to execute the tests.

Tangent – Generating the Katalon command line string is beyond easy.

katalon.png

Installing Katalon on each Windows slave machine and remembering to do so in the future is a pain.  Well, as of Katalon version 4.8, you can now execute the tests from a Linux server via the Linux shell.

So, the solution I am thinking here is the following:

  1. Create a new docker image with Katalon installed and an ENTRYPOINT of ./katalon.
  2. Setup a Linux server as a Jenkins slave
  3. Install Docker on this new Linux server
  4. Pull the Katalon docker image to this new server
  5. Create a smoke test Jenkins job that will pull the smoke test files from Git down to the slave, and then run the docker container with the correct smoke test syntax included in the command. (Similar to this)

The benefits of this solution are numerous. (e.g. easily update Katalon versions, identical Katalon configurations between Linux slaves, easily recover from outages, etc…)

katalon

MSBuild Task – Execution Time

Looking to find out how long it tasks to execute a MSBuild task?  Below is a quick method that I recently used to accomplish this.  (Based on this blog)

  1. Before the task, you will need to set a property that is equal to the current UTC ticks.
  2. After the task, you will again need to set a property that is equal to the current UTC ticks.
  3. Now you will need to subtract #1 from #2, which will give you the number of ticks that task executed for.
  4. Convert the number of ticks to a friendly time value

Note – There are ten million ticks in a second

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" DefaultTargets="Init">
 <Target Name="Init">
   <!-- Set start property -->
   <PropertyGroup>
     <StartUTCTicks>$([System.DateTime]::UtcNow.Ticks)</StartUTCTicks>
   </PropertyGroup>

   <!-- Command to simulate sleeping for 4 seconds -->
   <Exec Command="ping 127.0.0.1 -n 5 -w 1000 >NUL"/>

   <!-- Set the finish time, calculate overall time, and then output in freiendly format -->
   <PropertyGroup>
     <FinishUTCTicks>$([System.DateTime]::UtcNow.Ticks)</FinishUTCTicks>
     <ExecutionTicks>$([MSBuild]::Subtract($(FinishUTCTicks), $(StartUTCTicks)))</ExecutionTicks>
     <FriendlyTime>$([System.TimeSpan]::FromTicks($(ExecutionTicks)))</FriendlyTime>
   </PropertyGroup>

  <Message Text="Init Task Total Time: $(FriendlyTime)"/>
  </Target>
</Project>

Below a the results with the execution time displayed in the standard format.

msbuild_1.png

Visual C++ Redistributable and Missing MSVCR100.dll

So you have encountered an error similar to the following when running a new program on a Windows computer.

  • The Program can’t start because MSVCR100.dll is missing from your computer

You will typically get this error when the program you are running was built with some version of Visual Studio C++ (VC), and that VC run-time version and its components are missing from your computer.

To view the current Visual C++ run-time components (Visual C++ Redistributable) installed on your computer, just view your installed programs from the control panel.  You should see something like the image below.  These different versions can run side-by-side without issue, so don’t be alarmed if your list looks similar to mine.

VC.png

If you are unsure of what version you need, you can download a tool like Dependency Walker and use it to view all dependent modules of your .exe file.  In the example image below, the VC run-time version your program requires will be VC12, which is the 2013 version of the Redistributable package.

VC1

Quick reminder of how the .dll names map to the versions of Microsoft Visual C++ Redistributable.

Visual Explanation of Future Updates – In the installed program image above you will see both 2015 x64 and x86 installs of the Redistributable.  Now look what happens after the 2017 x64 install, the 2015 x64 entry was removed. (In place upgrade)

VC2.png

VSTS Bot – First Thoughts

So I recently read about the new VSTS Slack Bot.  This type of automation really gets me pumped as it aligns with one of my core DevOps principles, automation!!

So it pretty easy to setup.

  • Hit this page and click the Add to Slack button.
  • You will then need to get your access code from with your VSTS account and type it into the Slack Bot

From there you can do things like view\create a Release by typing “Releases” and then clicking the “Create” button. (GH-LunchMenuAction = My Release for my Google Home Action)

slackvstsbot2.png

You can also view approvals and launch builds.

slackvstsbot

Kind of a cool little add-on from Microsoft.  It’ll will be interesting to see what functionality is added in the future.

Note – Stuff like this is why I would recommend going with VSTS and not TFS on-premise.  Eventually VSTS will have all TFS on-premise features, plus its cheaper and you get updates hassle free….  Its only a matter of time.

Git Command Helper for TFS Users

The majority of the projects I have worked on in the past have used either TFS (TFVC) or Git.  For the projects that have used Git, I’ve noticed that for the people unfamiliar with Git, there can be some hesitation because of the unknowns.

So in this blog post I will simply try to align the standard TFS commands with the corresponding Git commands.  Hopefully this will help out any beginner Git users who are already familiar with TFS.

  • TFS (Get Latest) – Git (Clone and Pull)
    • Git Clone makes a Git repository copy from a remote source.
    • Pull gets files from the remote repo and merges them to the local repo.
  • TFS (Add) – Git (Add)
    • Remember, with Git you are working with a local repository, so the “git add” command will add the file(s) to your local repo for tracking purposes.
  • TFS (Check-in) – Git (Commit) (Push)
    • The “git commit” command will commit your changes to your local repo.
    • The “git push” command will push the changes to the remote repo.
  • TFS (Check-out)  – Git (-)
    • Kind of like how you can setup TFS to auto check-out a file, Git automatically recognizes changes to tracks files
  • TFS (Branch) – Git (Branch)
    • The “git branch” command will create a branch as needed.
  • TFS (Label) – Git (Tag)
    • The “git tag” command will tag\label a Commit.
  • TFS (Shelve) – Git (Stash)
    • The “git stash” command will store\save\stash\shelve your changes.  Somewhat similar to the TFS shelve-set feature
  • TFS (Merge) – Git (Merge)
    • The “git merge” command merges changes from 1 branch to another similar to TFS and push to local repo.

I am by no means a Git expert, but hopefully this blog will help out any TFS guru’s uneasy about working with Git.

github-octocat