Gradle – Flat Repo (flatdir)

What the heck is a Flat Dir repository in Gradle?  Well, its basically a directory(s) that contain a set of files that can be used to resolve dependencies.

  • Note, Gradle will assume  no transitive dependencies with this type of repo.

A good use of this repository type may be within a company that does not have its own internal repository and does not allow builds to resolve dependencies externally at build time.

So their Gradle build file might look something like this if they need to pull in specific jar versions from the lib folder at compile time.  Gradle will use the jar name from the lib folder for resolution.

repositories {
   flatdir { dir 'lib' }
}

dependencies {
   compile group: 'junit', name: 'junit', version: '4.12'
   ...
   ...
}

Now if they have a lib folder with jar files managed uniquely and they want to include all of them at compile time, then the Gradle build script might look like this.

dependencies {
   compile fileTree(include: ['*.jar'], dir: 'lib')
}

 

gradle

Jenkins Redundancy like a Boss

Here is one way to add some data redundancy to your Jenkins instance.  The solution below won’t auto recover, but the data itself should be safe if your active Jenkins master fails.

JenkinsBoss

The following are some key points not really mentioned in the beautiful Google Docs created chart above!

  1. A dedicated file system is created on the NAS that is attached to each Jenkins master server
    • The NAS has snapshot backups enabled for X number of days on this file system
  2. The same user and group is created on both master servers
    • The group ID must be the same on both master servers
      • groupadd -g <unique_id> <group_name>
  3. The new user’s home dir should be on the NAS under the dedicated file system and the user should own it
    • /etc/passwd – home dir
    • chown – change ownership
  4. Tomcat is installed from 1 master server to the NAS
  5. Jenkins is installed from 1 master server to the NAS
  6. Tomcat is only kept running from 1 master server
  7. A load balancer like HAProxy can be used for active\passive load balancing

Some other recovery recommendations.

jenkins_image.png

Ansible and Local Execution

How can I execute an Ansible task on the local Ansible controller server?

The key is to use the “delegate_to” or “local_action” key words on your task.

So lets say we want to use Ansible to monitor an end point and perform some action if the response is not equal to 200.

  1. Create task to check URL using the URI module and “delegate_to” or do step #2
    • —
      – name: Endpoint Validator
      uri:
      url: “{{ url }}”
      delegate_to: 127.0.0.1
      register: response
  2. Create task to check URL using the URI module and “local_action”
    • —
      – name: Endpoint Validator
      uri:
      local_action: url “{{ url }}”
      register: response
  3. Create task to perform action (e.g. fail) if response it not 200
    • —
      – name: Endpoint Error Action
      fail:
      when: response.status != 200

Ansible_Linux_www.hackthesec.co_.in_

Visual Studio Version Numbers

The Visual Studio version numbers and product naming convention has always been a little confusing to me.

Example: I have Visual Studio 2015 installed on my PC and the installation folder path is “C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\devenv.exe”.

The table below shows the Product Name and corresponding Version number.

  • Visual Studio 2008 (V9.0)
  • Visual Studio 2010 (V10.0)
  • Visual Studio 2012 (V11.0)
  • Visual Studio 2013 (V12.0)
  • Visual Studio 2015 (V14.0)
  • Visual Studio 2017 (V15.0)

There must be a good reason why they didn’t align the Name and Version…..

Interesting VS 2017 Installation Notes:

  • VS 2017 installation path is “C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\devenv.exe”
  • You can opt to install VS 2017 in a different location and drive, but note that most files will still be installed on the C:\ drive.
    • “About 75% of the files get installed to System folder by design. The other 25% gets installed to custom install path” (Source)

VSTS2

VSTS – GIT – JENKINS

Lets say I am in need of a private Git repo for a small 3 – 5 person team….  Where should I look?

For this blog post I looked at GitHub and VSTS.  Below is the pricing for both.

From the links above, you will notice that VSTS is the better deal.  Its basically free, plus it includes many additional features!

So, I know I can connect my Jenkins instance to GitHub, but can I connect Jenkins to my VSTS Git repo?  The answer is yes and it’s basically the same process as connecting to GitHub

  1. Create your public\private keys with ssh-keygen
  2. Add your public key to VSTS
    • vsts_ssh.png
  3. Create “SSH Username with private key” Jenkins credential
    • User private key from #1 here
  4. Use this new credential when configuring Git in your new job

Now you will probably want your new Jenkins job to launch when changes are made to the VSTS Git repo.

Below are some of the options you have with Jenkins and VSTS

I must say that Microsoft is really pushing out some cool products!

Cygwin and PIP Package Missing

I installed Cygwin today and forgot to install the PIP package for Python 2.7.

cygwin

Looking online, I found that you could install the PIP package by re-running the Cygwin installation .exe.  However, re-running the .exe took a LOT longer than I expected.

It is 100% faster downloading and executing get-pip.py from the following location:

Execute “python -m pip –version” to verify your version!

Flask-RESTful – Basic Authentication

I’m continuing to develop the REST API that will be used with the API.AI Webhook.  I decided that some sort of authentication is needed.

I played around with adding Basic Authentication to my API as API.AI supports this.  Below are the steps I took to get my authentication setup using Flask.  (Recommend reading Flask-HTTPAuth documentation)

  1. Include the necessary package
    • Flask-HTTPAuth==2.3.0
    • flaskauth
  2. Add get_password callback function. 
    1. @auth.get_password
def get_password(username):
    if username == 'devopsunleasheduser':
        return 'devopsunleashedpassword'
    return None
  3. Add error_handler callback function (Note “jsonify()” will need jsonify package) 
    1. @auth.error_handler
def unauthorized():
    # return 403 instead of 401 to prevent browsers from displaying the default
    # auth dialog
    return make_response(jsonify({'message': 'Unauthorized access'}), 403)
  4. Add login_required decorator to both classes to verify authentication before returning any info. 
    1. decorators = [auth.login_required]

 

Brian Harry – DevOps Presentation

I was fortunate to have had the opportunity to listen to Brian Harry speak twice this week. (At work and at MHTA)  I have been a big follower of Brian since the early TFS 2005 days, so I was pumped!!!

His talk was about DevOps and his VSTS teams journey.  Below are some of the highlights I wrote down from his presentations.

  1. His team always strives to find the root cause of issues… Always!!!
  2. His team works in 3 week sprints.
  3. It takes his team about 2 weeks to push out VSTS updates to all locations.
  4. They push releases internally 1st which is a lot quicker.
  5. They have feature teams, with each feature team responsible for all aspects of there code.  (e.g. Build, test, deploy, production support, etc…)
  6. Their feature teams have dedicated roles that are responsible for troubleshooting issues in production .
  7. Brian uses VSTS to help planning his farm tasks.
  8. You can not put a timeline on your DevOps journey.  If so, there is a good chance you will fail.
  9. He highly recommended the book “Drive” by Daniel Pink!
  10. Merging is really expensive
  11. They have release branches and that is about it.  (In addition to master\mainline)
  12. More “Shift Left” testing, which mean more unit testing.
  13. Feature flags are a huge part of DevOps

Overall I really enjoyed listening to him and was able to congratulate him on the success of TFS and VSTS.

VSTS-2015.png

Ansible – Dryrun and Limit Hosts

Here are a couple of useful Ansible playbook CLI switches I tend to use frequently.

Dry-run mode – Add this switch if you want to see what Ansible will do, without actually doing it.

  • –check

Limit target hosts – Add this switch to specify what host(s) you want to run the playbook against.

  • –limit
  • –limit “host1, host2, host3”
  • –limit “myGroup”

Ansible_Linux_www.hackthesec.co_.in_