What is the boundary of DevOps engineer job role?

From the "how it should properly work"/"how it was meant" perspective there is no real "DevOps engineer" because DevOps is a methodology/culture which emphasizes also on technical practices which should stretch across as many roles as possible and feasible.

How can we apply the CALMS (Culture, Automation, Lean, Measure, Share) mantra here? (Taking into approach that depending on factors like maturity, complexity and resources in the context, approaches need to be adapted to what helps to solve problems in an efficient and sustainable way)

Requirements engineer: can I automatically qualify maturity of requirements and track their delivery?
Software engineer: what is required to run my software in test either productive environment?
Architect: can I have a big picture based on live metrics?
Tester: can I create virtual temporary infrastructure to support execution of my code?
System administrator: how can I better organize my help desk / 1st user support and leverage self-help / automation solutions?

Somebody with a wish to focus on the methodology itself will be somewhere in-between all these guys and obviously become "jack of all trades, master of none".

Given this, my consideration how I would approach your customers' inquiries could be:

Please follow the instructions to setup VPN in this Wiki/code from this Git repo and if it's not working, document/show steps you have taken.
Please share more details on the context (here is not clear who owns which parts).
Let's allocate time to develop emergency plan for immediate action if required and also run scaling tests on a test system to model experiments, collect data and finally learn under which circumstances the performance problems appear. Then, we will know why it is happening. :-)
Follow from 3.? What does mean schedule from your perspective and if you mean regular execution, how often? So please let's go back to previous item :-)
Possibly, have you got more specs which infrastructure option (Cloud, on premises) should we use for that and what and what are you going to do? Here is a link to generic infrastructure planner, let's allocate time to discuss what is specific for Spark/Hadoop here and how will delivery to it look like.
If identity management is not part of our architecture, let's assess what are you going to do. An outcome could be that you include creation of users to your database scripts.

So the bottom line in DevOps culture regarding the role boundary is:

professional communication first
the boundary is to set boundaries inviting people to a professional conversation between adults, to get more conscious on both sides (!) about requirements, responsibilities and goals. Not just thinking in solutions and delegation but also make a plan and assess risks.

Therefore seasoned and successful professionals who are today maybe 50+ yrs old say sometimes jokingly "we have been doing DevOps from the very beginning" and partly, they are right.

To conclude and highlight and reflect on the real-world existence of such job-role, organizations who do not have a DevOps culture proceed to define DevOps roles as the following, for example:

DevOps engineer is an advanced systems administrator who can code up the infrastructure and automate it completely.
DevOps engineer is a tools administrator with coding skills to support the Dev team and ensure delivery towards production (CI/CD, deployment scripting but also Jira administration)
DevOps engineer is a developer focusing on including infrastructure elements into development, like containers and cloud APIs.

After all, many companies still look for the "full stack engineer" who should obviously become additionally the DevOps engineer (along with "AI/ML,  blockchain and foobar").

Finally, the role boundary, possibly not only in the DevOps context, but in this special context indeed very likely:

is also in your responsibility
may change between projects and employers or just from time to time.


Accessing the committer's name as a Jenkinsfile environment variable

I'm not entirely sure what you're trying to ask in your question, so I'm answering the question I believe you're trying to ask.

A build does not necessarily have a single commit author.  It has a list of committers, which can be empty or can contain many committers.  To create this list, you can use this snippet in a scripted Pipeline (note that you may need to disable the sandbox in order to use .collect()):

changeAuthors = currentBuild.changeSets.collect { set ->
  set.collect { entry -> entry.author.fullName }
}.flatten()

I use this with Git as my SCM; I'm not sure if it works with other SCMs.

See also the following questions on StackOverflow (some answers may be outdated):

How to get culprits or committers inside a Jenkins workflow with one or more SCMs
How do you pull git committer information for Jenkins pipeline


Multiple runners for all projects OR single runner per project

After some research i found out that shared runners are good for the projects that are very much the same, and specific runners are better for the projects that are not like any project.

Shared runners can eat all resources available in the end with its queue's which makes specific runners more interesting.

My choice now is more like a little bit of both, so shared for all small and likewise projects, and specific for the large new projects as the documentation recommends.

From Shared vs specific Runners:

  ... The main differences between a shared and a specific Runner are:

  Shared Runners are useful for jobs that have similar requirements, between multiple projects. Rather than having multiple
  Runners idling for many projects, you can have a single or a small
  number of Runners that handle multiple projects. This makes it easier
  to maintain and update them. Shared Runners process jobs using a fair
  usage queue. In contrast to specific Runners that use a FIFO
  queue, this prevents cases where projects create hundreds of jobs
  which can lead to eating all available shared Runners resources.
  Specific Runners are useful for jobs that have special requirements or for projects with a specific demand. If a job has
  certain requirements, you can set up the specific Runner with this in
  mind, while not having to do this for all Runners. For example, if you
  want to deploy a certain project, you can setup a specific Runner to
  have the right credentials for this. The usage of tags may be
  useful in this case. Specific Runners process jobs using a FIFO
  queue.

  A Runner that is specific only runs for the specified project(s). A
  shared Runner can run jobs for every project that has enabled the
  option Allow shared Runners under Settings ➔ CI/CD.

  Projects with high demand of CI activity can also benefit from using
  specific Runners. By having dedicated Runners you are guaranteed that
  the Runner is not being held up by another project's jobs.

  You can set up a specific Runner to be used by multiple projects. The
  difference with a shared Runner is that you have to enable each
  project explicitly for the Runner to be able to run its jobs.

  Specific Runners do not get shared with forked projects automatically.
  A fork does copy the CI settings (jobs, allow shared, etc) of the
  cloned repository.

Still i would like to know everyone's mindset about this, as im still interested about your way of doing it!


What is the difference between using brackets (`[` `]`) and just specifying the command in the CMD option in Dockerfiles?

CMD with brackets is contained within an array. Without brackets, the command is executed as /bin/sh -c mysqld.  Best practice afaik is to always use brackets.


There really is no way to take on your problem with git, which is to deny a developer with access to the code (required to work with it) the possibility to steal the code. This is because as soon as your developer has the code in a readable format (to program on it), he can do with it whatever he wants, anyways.

git-crypt has a totally different use, and is not related to your problem at all.

To solve your problem, there is basically only one possibility: lock the network in which the code resides down completely - make it so that there is no physical way to transfer the code to the outside. This could mean:

No direct internet connection, obviously.
No http/https proxy either, no mail clients etc.
No way to add new devices to the network (unless through official ways). E.g., no unused network ports to plug anything in; no guest DHCP; MAC address filtering; no WIFI, smartphones; etc.
No way to plug an USB device or other transfer mechanism into any devices.

This may sound harsh, but really is not. It could be as simple as keeping your code strictly within a set of VMs, accessed by something like Vmware Horizon or something like that, with cut&paste and file transfer disabled, of course. Be nice to your developers and allow them to transfer stuff into the environment (their favourite editor, for example).

The measures so far do not require any physical changes, i.e. your developers still can work remotely or in their usual unsecure office. Obviously, they still can go to extreme measures (screenshots on their client PC...) to get specific parts of the code page-by-page.

To go a step further, in increasing severity (although we are going into James Bond territory here...):

Strict policy with severe punishment. Setup a specific contract which each emplyee which contains clauses as harsh as the laws of your country allow.
Switching access to the limited network to a physical area in your office.
Security measures like locked doors accessible to only specific personnel, video cameras with 24/7 human supervision and so on.
2-factor access (either 2 persons required to checkout code, or at least a physical TAN generator or something along those lines).

Note that in your case you want to make it hard for a thief who knows that he is doing something illegal and already has committed to doing a crime. He will not respect any "weak" protection (like rules/laws) anyways.


Are git pre-receive/update hooks serialized?

In general git allows concurrent operations because one of them will eventually fail. Some specific database updates will have file locks (like updating the index files) but multiple receives can be in flight at once I think. In this case whichever verification completed first would be allowed in, and the second would fail due to having an invalid history (i.e. not being a fast-forward push).

In general this kind of operation is not well suited to git's server-side hooks since if something does fail, the person will then have to do some surgery on their branches to get things back in a valid state. These kinds of checks are generally run on the workstation side, though then you run into the "fun" that is distributing git hooks as they can't actually be checked in directly.

Another common solution is to have a CI service like Jenkins running verifications (tests) and updating a second branch or a utility tag showing which commits are acceptable.


to solve, after make all setup to Rundeck know and access de node you need go to Project Setting > Edit Configuration > Default Node Executor and change to SSH. The problem occured because my project was created like LOCAL default node executor.


Is AWS Fargate suitable for web applications?

Fargate is absolutely suitable for web apps. We have many customers that use it that way (and many others that use it in a more "batchy" way). Fargate is a managed compute engine and AWS takes a more responsibilities in terms of scaling, patching, security, and so forth so comparing raw Fargate costs to raw EC2 costs isn't how I would compare them. Having that said, raw EC2 capacity (like for like) is usually just 20% cheaper than raw Fargate capacity. If you factor in the cost of operations (less with Fargate) and the fact that you hardly drive EC2 clusters at full utilization, the comparison tends to skew towards Fargate most of the times. These are a couple of blogs that talk about this (blog one and blog two).


How do a flaten a list mixed with nested dict in ansible playbook?

I probably should use comment instead group the value in a dict who's value is a list

like

packages:
  - apt-file
  - axel
  - ibus-rime
  - rofi
  - i3
  - ffmpeg
  - xclip
  - cargo
  - httpie
  - ttf-ubuntu-font-family
  - tldr-py
  - glances
  # browsers:
  - brave-browser
  - chromium
  # editors
  - emacs


An idea key to your question is that of immutable infrastructure, which is the idea that an image is built once, deployed many times, and never changed at runtime. If the contents of the image need to change, an entirely new image is built from scratch and new instances replace the ones running the old image.

This is the opposite of what you might now call mutable infrastructure, where often a machine (or container) is booted from a generic OS image and then its contents managed by a configuration management tool on an ongoing basis. In this model, instances generally live longer and are changed in-place to respond to new requirements.

Packer is intended as a building block for immutable infrastructure. It is intended to orchestrate the creation of customized images that boot immediately into the desired state. If immutable infrastructure is the right fit for your use-case (which is not necessarily the case!) then indeed tools such as Ansible, Chef and Puppet may be unnecessary.

However, running these tools in a "one-shot" mode (e.g. Chef Solo) with Packer can often be useful for non-trivial images: these tools generally provide more powerful abstractions than Packer alone provides, making it easier to do things such as configure the init system to run certain services, or create user accounts. In this case, the configuration management tool would run as a provisioner in Packer, creating the necessary configuration within the image, but is not involved in the ongoing upkeep of instances booted from the image.

Most practical systems include a mixture of mutable and immutable infrastructure, since e.g. immutable infrastructure is inappropriate for stateful, centralized systems. In that case, using a traditional configuration management system can be useful to create a common base of functionality between your immutable and mutable systems, applying configuration in a one-shot way for immutable images and in an ongoing, stateful way to long-running, mutable machines.


How can I autoscale instances based on Kafka topic lag?

CloudWatch is absolutely the way to go on this front, you can surface Kafka Metrics in CloudWatch either by:

Having a separate process that pushes the metrics into CloudWatch.
Have your Producers, Consumers or Stream Processors push the metrics you need into CloudWatch.

The process you follow will broadly be broken down into:

Publish Custom Metrics.
Create an AutoScaling Launch Configuration.
Create an AutoScaling Group and Policies based upon your Custom Metric.

Depending on your cost model I would also consider how you can use spot instances to keep the costs down, i.e. if you have non-critical but high compute workloads use Spot Instances as part of your autoscaling configuration.


How can I force Docker Compose to rebuild containers when dockerfile changes?

Why not docker-compose build ?

https://docs.docker.com/compose/reference/build/

You can also do docker-compose up --build to force a rebuild.


CICD using jenkins which is installed in other machine

If your question is can Jenkins deploy to machine B then the answer is absolutely!  Jenkins provides documentation on deployments here, and has an example of a multibranch pipeline deployment here.

If you want machine B to deploy to a different computer (e.g. machine C) then you can do this by configuring machine B to be an "agent", and following the deployment guides linked to above.


How to automate Powershell's interactive Get-Credential method?

Your python problem is the \n after the commands, write already append a newline, so your script does the following:

$cred = Get-Credential # Start the command
# This line is ignored as extraneous after Get-Credential, but you can see it in your output.
Administrator # Send proprely to the User: prompt
# Sent to the Password: prompt
Password # Send to the shell input, causing the error at end of your log.

Now that's a quite hard way to automate the credential object in powershell, usually the object is build by other way like described in this blog post (quoting here for completeness):

$password = "mypassword" | ConvertTo-SecureString -asPlainText -Force
$username = "nwtraders\administrator"
$credential = New-ObjectSystem.Management.Automation.PSCredential($username,$password)

And used as such:

Get-WMIObject win32_logicaldisk -ComputerName Server1 -Credential $credential


How to clean up old images in docker registry with jenkins

We have a script which we run periodicaly which scans our docker registry for orphaned items, and then deletes them.

Unfortunately, it requires multiple API calls to get all the data required, and then to delete each image.  But basically, if an image hash is lacking a 'latest' tag or a tag that starts with 'v' it gets culled.

You could probably also try deleting based on last modified date.

We run this as a cron job in kubernetes, but you could just as easily run a script like this as a periodic jenkins job.


npm not found but installed from the shell script file in Dockerfile

The issue is that prereqs-ubuntu.sh uses bash to install the npm. While the RUN directive uses sh to run the commands.

Where is the npm installed

root@2cd4a6af90f4:/app# type npm
npm is /root/.nvm/versions/node/v8.16.0/bin/npm

The PATH for the RUN directive

# echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin


How to deploy my Express/React app to server with docker-compose and Dockerfiles

I found the solution, eventually it was just a volume issue, I had to add a specific line for modules :

volumes:
      - ./server:/app/server
      - /app/server/node_modules

After that I no longer got the "Can't find module" error
Still not sure what the impact of this is though


Nginx-Ingress Controller are compatible with Network Security Policy or not?

I don't think it is the compatibility, need to really see your configuration to be sure. But most likely, you block something wrongly. If you use the default network policy api networking.k8s.io/v1 notice that it only allows what is defined.


Kubernetes: Custom second scheduler container unable to start

In your manifest, you should set the spec.containers[0].livenessProbe.httpGet.scheme to HTTP not HTTPS.
This is because you use the option --port=10251. As the documentation said, you should not use the deprecated --port option anymore, because it expose the API insecurely.
You should set the following options: --port=0 --secure-port=10251 and then you would be able to set the spec.containers[0].livenessProbe.httpGet.scheme to HTTPS.


Are you basically saying that it’s hard coded to “localhost” and you need to somehow change the meaning of “localhost” because you cannot change the source code?

If so, it sounds fairly hard to achieve... but maybe you can override the meaning of localhost in the hosts file of the pod (the right way).

https://kubernetes.io/docs/concepts/services-networking/add-entries-to-pod-etc-hosts-with-host-aliases/

I feel like this will probably have unintended side effects and may affect kubernetes, but I could be wrong.  It’s definitely worth a try.

Note: the overrides can point to services created over the pod deployments if it works :).  They can have reliable internal domain names that don’t move around.


How to start a VPN connection using Jenkins?

solutions is: OpenConnect Plugin

See here: https://wiki.jenkins.io/display/JENKINS/OpenConnect+Plugin

OR also you can use below method:

On Linux (Debian based)

sudo apt-get install openconnect

Add the following lines to the bottom of /etc/sudoers (ubuntu configuration)

jenkins ALL=NOPASSWD:/usr/sbin/openconnect*
jenkins ALL=NOPASSWD:/bin/kill*

Then you will get an option in the Jenkins run task called:

Connect to Cisco AnyConnect VPN


How to consolidate multiple servers for CPU and Memory with Kubernetes

Since Pod is running on single node it can use resources from this node only. And CPU/MEM can't be shared here. At least without special software for distributed computation.
In your case you should consider using ActiveMQ Clustering (https://activemq.apache.org/clustering). In that way you will be able to spread load across multiple k8s nodes with ActiveMQ pods running on them.
So use k8s Deployment + pod affinity to be sure pods will be scheduled to different nodes.


Istio: envoy proxy to https container in same pod

Changing the destination rule from ISTIO_MUTUAL to SIMPLE fixed this issue.
There's obviously a problem with my config with the mTLS.


Apache container cant access php files mounted in /var/www/ - 403 error

SELinux prevent the docker filesystem to work in the intended way, to fix this issue just add the :Z flag at the end of the volume declaration. It goes something like this:

docker run -d -p 8080:80 -v {my_php_app_path}:/var/www/html:Z nimmis/apache-php7


What are the best practices of merging dev to master?

You can use "git flow" or more simple "gitlab flow".

Our small team merge into master on release or sprint ends.


DevOps Newbie - How to automate windows infrastructure deployment?

The first thing I can recommend for you is to remove all of the manual steps in your deployment pipeline. DevOps is all about automating as much as possible to allow for fast, frequent, and safe deployment of your infrastructure.

Second, if you haven't already, I would recommend using a build and deployment tool (Jenkins, Bamboo, GitLabCI, etc.) to compile your binaries and execute your automated deployment/configuration scripts. Having a central location for your deployments allows you to be able to scale more effectively, avoid repetition, and monitor your deployments more accurately. You should also store ALL of your scripts in some form of revision control ASAP (if they are not already).

Finally, I recommend you read up more on DevOps best practices and how it can affect your organization. In order for DevOps to be truly effective, your entire development team and company must adopt best practices and be 100% on board with a shift in how you manage your deployments.

Some reading I can recommend are:

The Phoenix Project by Gene Kim, Kevin Behr, and George Spafford

A telling of a fictional, but very realistic story of DevOps transforming a company and the opticals that the team faces to get there.

The DevOps Handbook by Gene Kim, Patrick Debois, John Willis, Jez Humble, and John Allspaw

Outlines the steps/stages to transform your company into a DevOps organization along with an assortment of real life DevOps transforming case studies.

Accelerate by Nicole Forsgren, Jez Humble, and Gene Kim

A plethora of empirical data and analysis on how DevOps has improved organizations both within IT and the entire company.

Continuous Integration by Paul Duvall, Steve Matayas, and Andrew Glover

Lays out the steps to take to move your organization to be able to deploy to the highest standards possible.


Minikube for Ubuntu 20.04 WSL stuck on 'Restarting existing docker container for "minikube" ...' after minikube start and 'StartHost failed' error

Not sure if complete reinstall is a valid option for you, but for me this helped:

sudo minikube delete
After delete is finished, minikube start again


Meaning of metrics in Google Cloud Console

It seems that my cluster is capped at 3% of maximum utilization, if I
  read the units correctly. This makes no sense at all, why isn't it
  capable of reaching 100% ?

Why do you think it does make no sense at all?

  I ran a few tests that created quite a few pods

What is a few tests?

  (each pod running basically an hello world script)

What does this script do? Print helloworld? If that is the case then it could be possible that the cluster will not use 100%.

By the way, how large is the cluster? How many agents? How many memory? So if the cluster possesses 1024GB of memory then it could be the case it is using 3%.

In summary, to indicate whether something makes no sense at all a number of facts is required:

content of the helloworld script
characteristics of the cluster, e.g. totol cpu, memory, type of disks, storage capacity, number of agents, masters
definition of few tests and few pods. How many tests were run, frequency, e.g. 10/s, how many pods

If you would really like to loadtest a k8s cluster, kubernetes has some documentation about this, e.g. this document explains how to load test a k8s cluster and in this case force autoscaling.


Scheduling a Docker Container or Installing Scheduler in Container

You don't need cron. Most server OS-es (CoreOS, RedHat, CentOS, Ubuntu, etc.) these days run systemd, and systemd unit files can run your container to execute the job, for example:

Two unit files. One is mytimedstuff.service and mytimedstuff.timer. The .timer defines when the service should be executed.

In mytimedstuff.service

[Unit]
Description=Executes mystuff
After=default.target docker_network_apps.service
Requires=default.target docker_network_apps.service

[Service]
Type=oneshot
User=root
ExecStartPre=/usr/bin/docker pull mystuff:stable
ExecStartPre=-/bin/bash -c "/usr/bin/docker rm -f mystuffcontainer 2>/dev/null"
ExecStart=/bin/bash -c "/usr/bin/docker run --name mystuffcontainer mystuff:stable mystuff"

In mytimedstuff.timer:

[Unit]
Description=My stuff runs at 00:10 sharp.

[Timer]
OnCalendar=00:10:00

[Install]
WantedBy=multi-user.target


Pre-install Jenkins plugins using Docker image

This seems to be the right idea: https://stackoverflow.com/questions/29328278/installing-jenkins-plugins-to-docker-jenkins/29328489#29328489

so in the Dockerfile you'd just use:

COPY plugins.txt /usr/share/jenkins/plugins.txt
RUN /usr/local/bin/install-plugins.sh < /usr/share/jenkins/plugins.txt

I tried it and it worked for me.


How do I make ansible-playbook log its output in a machine readable format like xml or json?

This is done using one of the configuration options. For a general overview of configuring Ansible, see the configuration documentation, in particular the callback plugin configuration.
Callback plugins change how Ansible provides feedback on the progress and state of tasks. There are several options for JSON output:

JSON output to stdout
JSON to syslog

Setting these configuration options can be done either in a configuration file (ansible.cfg) or with environment variables (ANSIBLE_XXX), depending on what you prefer.
To quote the current documentation:

Changes can be made and used in a configuration file which will be searched for in the following order:

   ANSIBLE_CONFIG (environment variable if set)
   ansible.cfg (in the current directory)
   ~/.ansible.cfg (in the home directory)
   /etc/ansible/ansible.cfg

Ansible will process the above list and use the first file found, all others are ignored.

Assuming you want to send the output to stdout then, your ansible.cfg could look like:
[defaults]
callback_whitelist=json
stdout_callback=json

Using environment variables:
 ANSIBLE_CALLBACK_WHITELIST=json ANSIBLE_STDOUT_CALLBACK=json ansible-playbook ...


Is Hashicorp's Vault suitable for personal projects as well?

Vault will distribute multiple unseal keys but there's nothing to stop you from keeping all of the keys in one place for a personal project.

The one time I did it I set the keys to 3 and kept them in different physical locations along with online locations.


How to make .DACPAC from Azure Sql Server using Azure DevOps release pipeline?

Ok, after researching some more time, I've found this gem https://markheath.net/post/backup-restore-sql-database-azure-cli. In this article author uses az sql db export and az sql db import to save database into BACPAC and restore it back on the clean database. But here comes a pitfall - Azure CLI task v1.0 uses CMD so it's not easy to translate from Bash syntax to CMD, and Azure CLI task v2.0 preview which has option to choose Bash as an interpreter doesn't have az installed (it's probably a bug and already has a post on uservoice) so one needs to rewrite the script into Powershell if they want to use it.
So, it's not completely what I was looking for but it does its job.


"Error: forwarding ports: Upgrade request required" Error in helm of a kubernetes cluster

See my answer here:

Ran into this today when trying to use Garden.io for a cluster running in Jelastic.
Found the solution in this Github comment:
First acquire a local binary for Tiller (server-version of Helm), either by compiling or by downloading it from the release page.
Then run:
$ export HELM_HOST=":44134"
$ tiller -listen ${HELM_HOST} -alsologtostderr >/dev/null 2>&1 &

This will run a local version of the Kubernetes Helm Server. Now try your original command again, kubectl, that will delegate to this local Helm instead and manage to connect.


In Jenkins when to use SSH site and when to use a node?

If I need to build, run checks, tests, ... I use a node.

If I want to open a connection to a server to deliver the built code to production for example, I use the sshCommand step from SSH Step plugin. When the code is delivered the ssh connection is closed.

Having a node always connected to your jenkins can be unpleasant. Let's say the server you connect to belongs to a partner and he wants to audit the times you connect to put the files on the server. In that case he cannot say. Having a node always connected also means you need to maintain the connection, imagine the server is configured to disconnect you after a while or the server reboots and the connection is not relaunched.


Transfer files and execute commands from jenkins scripted pipeline to AWS instance

Below is sample groovy code for sending file or execute command over SSH in jenkins:

node(master)
{
  stage(Deploy)
  {
     sshPublisher(publishers: [sshPublisherDesc(configName: 'SERVER_NAME', transfers: [sshTransfer(cleanRemote: false, excludes: '', execCommand: 'apt-get update', execTimeout: 120000, flatten: false, makeEmptyDirs: false, noDefaultExcludes: false, patternSeparator: '[, ]+', remoteDirectory: '', remoteDirectorySDF: false, removePrefix: '', sourceFiles: '*.war')], usePromotionTimestamp: false, useWorkspaceInPromotion: false, verbose: false)])
  }
}


What's the relevance of periodicity in CI / CD processes?

I would separate the CI and CD contexts, as the periodicity in one of them is rather loosely coupled to the periodicity in the other one.

That's primarily because CI attempts to produce versions of software available for delivery/distribution, see How does continuous integration relate to continuous delivery / deployment?
But not all CI executions are successful. CD periodicity will, at best, be equal to the CI one, but typically it will be lower.

In CI the periodicity is tightly coupled to the desired development velocity (and maybe branch stability) and can be driven by different project-specific requirements. Here are just some examples of CI execution patterns, there could be others:

launched for every changeset commit - the recommended one, as it offers the shortest time to identify a changeset causing a regression, see Build on each commit - Continuous delivery. Costs vary with the commit rates.
launched every N (fixed number of) commits - trade culprit identification performance of #1 for slightly lower costs (overall fewer executions)
launched every T (fixed) time interval

fixed costs due to predictive amount of executions: no cost hikes for commit activity peaks, a day with twice the number of commits than another will still cost the same
culprit identification performance drops vs #1 if multiple changesets are committed within T
consecutive executions without any changeset committed in between offer measurements of the verification process reliability - different results in such executions could indicate unreliable verifications

launched whenever resources for execution are available - if that's where the bottleneck of the process lies. Any of the above categories can appear depending on the pattern of commits relative to the intervals between executions. This maximizes the verification resources ROI.
launched with one pattern during working hours and a different pattern overnight - to balance computing resources utilisation, for example.

Another possible reason for periodicity mismatch between CI and CD is the variability in the duration of the CI executions. Caused, for example, by that prohibited build schedule policy you mentioned (see also How to implement a frozen test environment?) Or by builds executed on classes of servers with significantly different performance, thus lasting different amounts of time.

Even for a successful CI execution in which the produced software version qualifies and is available for deployment it doesn't necessarily mean it will also be immediately deployed (or even deployed at all). There can be other criteria or policies preventing it (business reasons, for example).


How do i print the list of jobs per plugin in Jenkins

They quick and dirty answer is to "Display jobs group by the build steps they use" as updated below.
import hudson.model.*
import hudson.tasks.*

//All the projects on which we can apply the getBuilders method
def allProjects = Jenkins.instance.allItems.findAll{ it instanceof Project }

//All the registered build steps in the current Jenkins Instance
def allSteps = Builder.all()

//Group the projects by the build steps used
def projectsGroupBySteps = allSteps.inject([:]){
   map, step ->
   map[step.clazz.name] = allProjects.findAll{it.builders.any{
        it.class.name.contains(step.clazz.name)}}.collect{it.fullName}
   map
}

//Presentation
projectsGroupBySteps.sort().each{
    println "--- ($it.value.size) $it.key ---"
}
println ''

println 'Occcurences:'
projectsGroupBySteps.sort().each{
println "--- ($it.value.size) $it.key ---"
    it.value.each {
        println it }
    println ''
}
return

Note that a job has multiple sections: builders, publishers and buildWrappers.
Depending on what you are looking for, replace
[allSteps = Builder.all(), allProjects.findAll{it.builders.any ]
[allSteps = Publisher.all(), allProjects.findAll{it.publishers.any ]
[allSteps = BuildWrapper.all(), allProjects.findAll{it.buildWrappers.any ]

Plugins may also be involved in the SCM section and as properties (eg: hudson.plugins.buildblocker.BuildBlockerProperty, com.sonyericsson.jenkins.plugins.bfa.model.ScannerJobProperty)
Obviously, same limitations of plugin-usage apply: does not scan pipeline steps plugin usage or build steps nested within an org.jenkinsci.plugins.conditionalbuildstep


When configuring PM2 is finished and it's working properly, you must run two commands to make your PM2 as a autostart service and enjoy that.
1) Run 'pm2 save': It's save all of your PM2 configure to a file.

admin@server:~$ sudo pm2 save

2) Run 'pm2 startup': This command creates a system service and runs your last change saved with the first step.

admin@server:~$ sudo pm2 startup

So, Run the first command when you changed the configuration to store the last changes.


Updating Opera browser through command line (Windows)

One you use chocolatey to install Opera. Upgrading is also supported:

choco upgrade opera


This error CannotPullContainerError usually occurs when you have no access to the Internet from ECS and thus it doesn't able to pull an image from registry.

Make sure you have networking configured in such a way to have an access to Internet https://stackoverflow.com/questions/48226547/aws-fargate-cannotpullcontainererror-500

Manually you can confirm everything with such steps

Find route table. Check whether it belongs
to your VPC.
Find proper NAT (you need NAT Gateway ID) (IMPORTANT: NAT should be launched in PUBLIC
subnet)
Create a route for that

$ aws ec2 create-route --route-table-id "rtb-037148d7b5967a231" --destination-cidr-block "0.0.0.0/0" --nat-gateway-id "nat-0c137ae8a2b409088"

You can also use internet gateway instead of NAT but the main idea is the same - to create connectivity from service to registry.
If you have that in place and verified you can easily rewrite that to ansible tasks.

Update
To start an ECS task you want to use ecs_task module https://docs.ansible.com/ansible/latest/modules/ecs_task_module.html

To pull image from repository and start it you can use these tasks

- name: docker login
  shell: "$(aws ecr get-login --no-include-email --region {{ default_region }})"
  args:
    executable: "/bin/bash"

- name: pull latest app image
  docker_image:
    name: "{{ ecr_repository }}/myapp:{{ image_tag }}"
    force: yes

- name: run application with docker compose
  docker_service:  # module is called docker_service before Ansible 2.8
    project_name: "myapp"
    project_src: "{{ app_dir }}"  # path to a directory containing a docker-compose.yml
    pull: yes
    recreate: always  # run scheduler from a clean state
    remove_orphans: yes  # remove containers for services not defined in the compose file
    state: present  # specifying present is the same as running docker-compose up
    restarted: yes  # use with state present to restart all containers
  register: output

but in case of Fargate the above is not needed as everything is handled by task definition


What is the best way to find the Auto Scaling Group of an ECS cluster?

Example in AWS cli that should work to get the ASG name

a=curl 'http://169.254.169.254/latest/meta-data/instance-id'; aws autoscaling describe-auto-scaling-instances --instance-ids $a --query 'AutoScalingInstances[*].AutoScalingGroupName'


Problem on gitlab-runner: cannot lock ref [...] Another git process seems to be running in this repository

The easiest solution for me was to execute

docker container prune

on the runner machine. The command removes all stopped docker containers and the .lock files therein. Running a pipeline from gitlab.com then starts a new container. (Note that all caches will be lost).


Automate Application Upgrade in Azure VMs

Do you think this might help you?
https://docs.microsoft.com/en-us/azure/automation/update-management/update-mgmt-overview#:~:text=You%20can%20use%20Update%20Management,and%20in%20other%20cloud%20environments.&text=For%20multiple%20Azure%20VMs%20by,page%20in%20the%20Azure%20portal.
If your VM is disposable and you can replace it with a new VM then you can also use VM Image galleryfor Linux where you can have images for a VM or multiple VM types and you can update it at one place using Azure DevOps and then update the underlying VM image
https://docs.microsoft.com/en-us/azure/virtual-machines/linux/image-builder-gallery-update-image-version


pyinfra was designed for this purpose. Fabric can also be used to script configuration.


Processing message queues quickly without wasting money on AWS Lambda

An SQS queue can now be wired directly to a Lambda function as an event source, so you don't have to start the Lambda function on a schedule.

https://aws.amazon.com/blogs/aws/aws-lambda-adds-amazon-simple-queue-service-to-supported-event-sources/


Running a command as a specific user on an ec2 using ssm

I didn't find anything on AWS or boto3 docs that allows for that, but I was able to execute as a different user using the runuser command. In theory, you could do the same thing with a combination of sudo and su commands, but this one is pretty simpler.
For that, you can do as follows:
runuser -l  userNameHere -c '/path/to/command arg1 arg2'

Since send-command executes as root, you don't have any issues.
Note: I thought that send-command uses in some way a session managed by the SSM Session Manager, but I was wrong. I spent a good time configuring SSM Session Manager preferences and tagging IAM resources according to this doc and this one, but send-command always execute as root as far I saw.
Sources:

https://man7.org/linux/man-pages/man1/runuser.1.html
https://www.cyberciti.biz/open-source/command-line-hacks/linux-run-command-as-different-user/


In what way does a new ECS Fargate service not satisfy the task definition's required capabilities?

This is by design because AWS have yet to upgrade LATEST to point to v1.4.0 (as per 2020-06-23, see AWS ECS Developer Guide). Set the service platform version explicitly to 1.4.0 (instead of LATEST).


Cheapest and easiest way to run a periodic Docker process

AWS offers fargate now, which is managed by them rather than being a service like ECS which you ran the servers for and they manage the control plane. Fargate is basically "serverless" ECS. You can use a cloudwatch scheduled even to run something occasionally on a schedule on a fargate ECS cluster. Therefore you're only paying for what you're using. Something to consider is fargate is quite a bit more expensive than ECS running on your own EC2 instance, but if you're running in frequently you'll likely see a pretty marginal savings.

Another option would be to automate something similar. So create an ECS cluster with an ASG backing it, and adjust scale to match ECS demand, and then use cloudwatch to launch the container. ECS can then scale up and down, but this can be tricky because this method is usually used to autoscale ECS clusters based on cloutwatch triggers, so you might have to hack around in cloudwatch and possibly hack together a lambda script, since you're technically not using it the way it's intended. In the end this would probably be the most cost effective, but in your case the cost savings could be negligible.

In my opinion Kubernetes is a ten ton hammer that you'd be using to hang a picture on the wall. It's way overkill.

You can explore those 2 options and see if they are cost effective, and easy enough to manage. Other container orchestration is usually going to be overkill, ECS has the advantage(usually a disadvantage) of being stupid simple and you pay very little and have basically no operational overhead for the control plane.


Is Elastic Beanstalk suitable for enterprise-grade CD?

IMO opinion your issue isn't with Elastic Beanstalk in that scenario, it's with Jenkins, or at least the way you're using it. You should really concentrate on building "a thing" only once, regardless of what that thing is.

Full Disclosure: I work for ThoughtWorks and am incredibly biased to GoCD. I'll try to spell out what I mean in as neutral was as I can. I will use our tool's docs are examples, but hopefully people can extrapolate to their systems.

Somewhere early in your pipeline you're building "artifacts". This could be binaries representing all or part of your application, or they could be output from any number of tools such as testing tools. These artifacts should be stored by the system and never built again. The system should then fetch the artifact from the proper revision when needed.

For example...

I build a .jar file and run some unit tests on it, basic C/I stuff. If it passes, that .jar file and the output of the tests gets uploaded to that specific pipeline job.
The next pipeline could be your deployment to a more complex testing environment. It should fetch the exact jar from the exact job that built it. It then executes Elastic Beanstalk to deploy that jar to the correct environment.
The next pipeline is your staging deployment. It goes all the way back to the first pipeline and fetches the exact jar from the exact job that built it. In then executus Elastic Beanstalk to deploy that jar to the correct environment.

These are each separate pipelines because that allows you to run more in parallel or on demand without blocking.

You could use Elastic Beanstalk, Chef, Puppet, Ansible, uDeploy or any number of other tools to do the actual deployments. That's not where your issue is coming from. Continuous Integration servers weren't built originally to do this. Of course there are lots of plugins you can use to get to the same place if that's your preference.

Continuous Delivery servers like GoCD, Chef Automate and ConcourseCI were built specifically to solve things like this.


How to integrate firebase functions into devops pipeline

I found the solution.

As build artifact you need to publish, firebase.json, .firebaserc and whole functions folder including package.json (without node_modules)

Then in release using sudo firebase deploy --token $(firebase-token) --project $(firebase-project) --only functions --force


Is there a [ci skip] option in GitLab CI?

Yes, there is:

If your commit message contains [ci skip] or [skip ci], using any capitalization, the commit will be created but the pipeline will be skipped.
Alternatively, one can pass the ci.skip Git push option if using Git 2.10 or newer:
git push -o ci.skip

From: GitLab CI YAML documentation - "Skip Pipeline"


When should an application be copied to a new region?

First a primer on regions and availability zones

Regions are physical places where the AWS resources are located.  As such different regions may offer different resources.  This is also important for latency-sensitive operations and data sovereignty laws.  Having an EC2 instance in the us-east-1 region means all of my instance resources are in that region of the United States.

Availability Zones, while in the same geographic area, may be backed by one or multiple data centers.  However, if you have an EC2 instance running in two availability zones, they will run in separate data centers.  So an EC2 instance in us-east-1a and us-east-1b will be in two separate data centers.

Imagine you have a mission critical application that needs high availability.  By setting up a load balancer (controlled by the region and not the availability zone) you can help scale your application across multiple availability zones and redirect traffic if something happens to an availability zone.  This helps create a highly available infrastructure.

To answer your three examples:

Computational availability is not really a factor.  You could always throw a larger instance (or multiple instances in the same availability zone) to decrease load.
While you could use multiple zones for rolling out upgrades, this can also be done within the same availability zone.
Your third point here is the main draw to the availability zones.  If a data center catches on fire, floods, gets hit by a meteor and all your EC2 instances are tied to that center then you would be unable to access whatever application you had running.


horribly insecure:
root $
root $ docker login
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
Username: nicksaunders
Password:
WARNING! Your password will be stored unencrypted in /root/snap/docker/471/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
root $
root $ docker images
REPOSITORY                   TAG                 IMAGE ID            CREATED             SIZE
helloworld_web               latest              3118807ab0e0        13 minutes ago      196MB
frappe/erpnext-nginx         latest              e417869e11ba        39 hours ago        414MB
frappe/erpnext-worker        latest              96559438dfec        40 hours ago        1.22GB
frappe/frappe-nginx          latest              3d006c375c6e        40 hours ago        410MB
frappe/frappe-worker         latest              961988528953        40 hours ago        1GB
frappe/frappe-socketio       latest              bca49f1b4a04        40 hours ago        180MB
frappe/bench                 latest              8c1bc9d3a572        40 hours ago        1.4GB
frappe/erpnext-docs-nginx    latest              7d117e2bd798        42 hours ago        428MB
frappe/erpnext-docs-worker   latest              a1c0b0511e92        42 hours ago        2.88GB
frappe/erpnext-worker        <none>              09be9dde8f8f        2 days ago          1.21GB
frappe/bench                 <none>              c775a49dee59        2 days ago          1.4GB
redis                        alpine              bd71e6db4a54        13 days ago         32.2MB
python                       3.7-alpine          295b051ee125        2 weeks ago         41.7MB
basex/basexhttp              latest              3339bf6bc898        3 weeks ago         226MB
twilio/twilio-java           latest              611ab7ece1cf        5 weeks ago         993MB
mysql                        latest              0d64f46acfd1        7 weeks ago         544MB
jetty                        latest              5f997007f18d        7 weeks ago         522MB
jitsi/web                    latest              1fad75476320        2 months ago        483MB
mysql/mysql-server           latest              8a3a24ad33be        2 months ago        366MB
hello-world                  latest              bf756fb1ae65        8 months ago        13.3kB
lukptr/erpnext7              latest              130f3d9f222f        14 months ago       3.59GB
root $
root $ docker tag helloworld_web nicksaunders/helloworld:firsttry
root $
root $ docker push nicksaunders/helloworld
The push refers to repository [docker.io/nicksaunders/helloworld]
fca41575e369: Pushed
40f0977ea685: Pushing [==================================================>]  12.08MB
f8b013a4d861: Pushed
1d6d0bace1b8: Pushing [====>                                              ]  13.03MB/143MB
aabee2d268fc: Pushed
22ee430ae506: Waiting
60faa6c61cf9: Waiting
cfd52085ef94: Waiting
408e53c5e3b2: Waiting
50644c29ef5a: Waiting

but seems to work.


How to do blue-green deployment with PL/SQL code?

Yes, it is possible, but it takes a bit of work. I assume that the concern with duplicating the database is 1) time and 2) size/storage, otherwise it seems like it would be trivial to spawn databases for testing against. Based on the fact that you are targeting PL/SQL i'm guessing you are working with Oracle.
You will either need: 1) a smaller-scale dataset that can act as a representative sample of you data which is sufficient for testing all of your corner cases or 2) to clone the whole production database. Honestly, I recommend option #2. There are numerous cases where things operate correctly at a smaller scale, but run into issues once scaled up - especially with database data. Ideally, you could store and replay a sample of queries for load testing using a database proxy like ScaleArc (if Oracle won't let you store your queries).
Saying you need to clone your whole dataset may make me sound like a kook and like it will take forever, but you may find this answer in which I detail a methodology for cloning multi-terabyte databases in seconds without significantly increasing your storage costs due to duplication and delta-snapshotting useful. This may be harder if you are using Oracle's proprietary storage system. Be forewarned, the tech isn't cheap, but if you wanted cheap, you wouldn't be using Oracle. The best way to sell that to your upper management is to prove that the tech is cheaper than the outage/data mangling it would prevent. It's your insurance policy against outages - so how much insurance are you willing to buy?


CIS Benchmark for Docker and Docker Bench for Security: why, how and when?

Larger organizations are concerned that containers introduce more security risks than traditional VMs as there isn't any hypervisor segregation. Developers running arbitrary images, that run as root, that they don't bother to security patch, looks worse for security team's than automatically patched VMs controlled by ops teams without containers.

Misconfiguration of docker demon itself can also introduce security risks. It's fairly common for complex software to have security tuning tools and to run them as part of ”operational readiness checks” of the live environment before going live as a security audit. Then also run them on some periodic cycle or after any major upgrades of the software that is being audited. The tools you name will be very useful for that purpose.

Software vendors who build container orchestrators are actively working in removing any dependency on needing to run docker demon see cri-o.io. Security, footprint and fork time are areas that vendors are competing on. CoreOS was a big innovator in this space before they were acquired you can look at some of their architecture presentations to get a flavor of what containers at scale workout docker looks like.

From a devops perspective we should empower devs to run security tools all the time to fix issues early. In practice devs might be using Docker for Windows locally but the live env would be a container orchestrator possibly running rkt. In reality devs in large organizations are not going to security scan infrastructure they didn't create and dont have root on. Even at a startup not every dev can be familiar with all of the tooling and software running in production. An external security audit is a smart investment.

IMHO if your teaching containers from a developer perspective it's worth pointing out such tools. Certainly devs in my experience are woefully unaware that running containers they find on docker hub as root isn't the smartest way to be secure. I would only expect to see such tools to be covered in depth by courses teaching security analysts how to audit systems that use Docker.


Can you inherit a template_cloudinit_config in another template_cloudinit_config object in Terraform?

The cloudinit provider only knows how to encode multipart MIME documents to pass to cloud init; there is no mechanism for parsing to modify and then re-encode.
However, a different way to frame your problem is of constructing a sequence of part objects in a local value and then passing them all together to the cloudinit_config data source, like this:
locals {
  core_config = tolist([
    {
      content_type = "text/x-shellscript"
      content      = file("${path.module}/scripts/core_script1.sh")
    },
    {
      content_type = "text/x-shellscript"
      content      = file("${path.module}/scripts/core_script2.sh")
    },
  ])
  custom_config = tolist([
    {
      content_type = "text/x-shellscript"
      content      = file("${path.module}/scripts/custom_script.sh")
    },
  ])
}

data "cloudinit_config" "core" {
  dynamic "part" {
    for_each = local.core_config
    content {
      content_type = part.value.content_type
      content      = part.value.content
    }
  }
}

data "cloudinit_config" "custom" {
  dynamic "part" {
    for_each = concat(
      local.core_config,
      local.custom_config,
    )
    content {
      content_type = part.value.content_type
      content      = part.value.content
    }
  }
}

resource "aws_instance" "vm1" {
  instance_type = "t2.small"
  ami           = data.aws_ami.ubuntu_ami.image_id

  # ...

  user_data = data.cloudinit_config.core.rendered
}

resource "aws_instance" "vm2" {
  instance_type = "t2.small"
  ami           = data.aws_ami.ubuntu_ami.image_id

  # ...

  user_data = data.cloudinit_config.custom.rendered
}

The above is using dynamic blocks to generate a dynamic number of part blocks based on the number of elements in a list, which means you can focus on making sure the lists have the necessary items, using whatever dynamic expressions you need to decide that, and have Terraform project that into zero or more part blocks automatically.
Note: the hashicorp/template provider has been deprecated and continues to be available only for backward-compatibility. For that reason, I wrote the above to use the hashicorp/cloudinit provider instead. Its cloudinit_config data source is based on what was formerly template_cloudinit_config.


Jenkins - Mono repository build only changed the folder

Am assuming you are familiar with Groovy at basic level. Well, you could do the following with Shell scripts too, but I have done something similar using native capabilities of Groovy.

Store changeset by running git log ..., those three dots mean you should go thru' git-log man page to check the options that will list the changed files with folder names.
Iterate thru' the changeset (from item 1) to identify the folder (microservice 2 or 3, 4, so on and so forth).
Once the service is identified, try and set few environment variables that can be reused in other scripts, if you want. Advantage with this approach is, you stand to have access to them until the completion of build.

Hope this helps.


I'm a little confused about the exact use case. Are you asking about writing to volumes that are mounted inside the container?

You can change what the process in the docker container runs as by using --user on your run commands.

docker run --user 1000 --ti centos/7 /bin/bash

This answer may help you:
https://stackoverflow.com/questions/41100333/difference-between-docker-run-user-and-group-add-parameters

The manual explains this as well: https://docs.docker.com/engine/reference/run/#user


As this question is quite high-level, I give a high level answer.

Technically, if there is an incident, you would normally enforce a shutdown of a running container and a replacement with a newer image with an implementation of the change request resulting from the incident.

How much automation will be possible to get there, I think it really dependends on single cases or at best their classes.

Best is therefore I think to assess different scenarios, group them into classes and derive solution workflows. Once a problem catalogue and a required set of organization policies are there you will be able to sort out which tools can help to realize them and which automations grades are possible.

For example:

discovered system environment level vulnerabilities are normally patchable through operation system security upgrades. You could have batch job delivering daily updated base images.
application environment level changes like "let's migrate from Java 8 to Java 9" is an organizational decision which requires an organizational process
application code level vulnerabitites requiring updating to patched versions requires some ways of dependency management governance in your organization, tools for quality gates are available but without an organizational decision nothing will happen here.
default credentials in production - again where do they come from? This is very special case and possibly a design failure; not sure how solve this in an automated way.

So I have to conclude that this question is more about culture than automation.


Running services on different ports in development environment and production environment

This should be fine.  It's something you may even have to do if ports are already in use on localhost.  Even development languages like .net core are now supporting different environments that can be used to configure the different ports.

Example: 1433 is the standard sql server port.  If I wanted to test a local sql server container, but did not want to stop my locally installed sql server, I could run it on another port (e.g. 1431).


Automatic Kubernetes deployments per Github pull-request

I don’t know of a specific tool but I have setup a lot of webhook handlers on OKD kubernetes to do git driven updates to kubernetes. All the scripts are on GitHub as OCD. The tools I used are:

adnanh/webook a go binary that you configure to catch git webhooks and run scripts. Here is my hooks.json config to match GitHub release events and run a build script on openshift kubernetes. Looking at the docs there are PR events you can match on. So you should be able to setup Webhooks to fire on PRs and trigger s2i builds.
hub is GitHubs official command-line tool. I put it into my tools image that runs my webhooks. It has some pull request commands. I haven’t checked them against your stated requirements. Hub is just a wrapper to their API so worst case you would need to use curl to update PRs.
Helm is the installer/template engine I use for creating apps. Helmfile is an excellent extension. So here is a generic “deployer” helm chart that can spin up a deployment. So OCD responds to GitHub events of new Helmfile declarative configuration being pushed and deploys or updated any helm releases modified in git.

For builds I had to do quite a bit to have an OKD container build tag the final container image with the GitHub release tag which is a generic “ocd-release-build” helm chart. I think that is necessary as each PR will need by tagged with the PR number in the container registry to deploy from that specific tag. I haven’t ever tried to get OKD to build a refs/pull but I would be shocked it didn’t work as expected.

So while OCD doesn’t yet spin up builds and deploys based on PR events it appears is likely that it could be extended to do that. We have migrated our apps from our in-house scripts over to OCD. A “PR previews” feature could be a very cool feature to add.  Feel free to raise a feature request.


Compatibility of http-builder-ng Groovy module with Jenkins 2

I would recommend against importing if at all possible. You can use built-in functionality to achieve the same results. Also be aware of "In-process Script Approval"

https://stackoverflow.com/a/42662243/1678094

// GET
def get = new URL("https://httpbin.org/get").openConnection();
def getRC = get.getResponseCode();
println(getRC);
if(getRC.equals(200)) {
    println(get.getInputStream().getText());
}

// POST
def post = new URL("https://httpbin.org/post").openConnection();
def message = '{"message":"this is a message"}'
post.setRequestMethod("POST")
post.setDoOutput(true)
post.setRequestProperty("Content-Type", "application/json")
post.getOutputStream().write(message.getBytes("UTF-8"));
def postRC = post.getResponseCode();
println(postRC);
if(postRC.equals(200)) {
    println(post.getInputStream().getText());
}

You can also override the request method:

https://stackoverflow.com/a/32503192/1678094

conn.setRequestProperty("X-HTTP-Method-Override", "PATCH");
conn.setRequestMethod("POST");


Incorrect timezoneID inside docker container

The case is dpkg-reconfigure tzdata simply creates /etc/localtime as a copy, hardlink or symlink (a symlink is preferred) to a file in /usr/share/zoneinfo. So it is possible to do this entirely from your Dockerfile. Consider:

ENV TZ=Asia/Kolkata
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone

And as a bonus, TZ will be set correctly in the container as well.

This is also distribution-agnostic, so it works with pretty much anything Linux.

The original answer: https://serverfault.com/questions/683605/docker-container-time-timezone-will-not-reflect-changes


How to automatically deploy routers / industrial PC's

Regarding no requirements on the target device, of the tools you mention, only Ansible does not need an agent on the host. The minimal requirement to use Ansible modules is Python on the host you want to update. If you don't have Python on hosts you could even send raw commands to the hosts.

Wikipedia has a list comparing configuration management software, there you can find other agent-less tools.

Not listed on Wikipedia is Fabric, a simple Python library to execute remote shell commands. It is agent-less as well.


How do I use docker command's "--format" option?

I take the risk of being downvoted here, but I find Go templates a nightmare to use beyond the most basic use cases. Don't take me wrong: they are extremely powerful as you can guess it by reading the docs, but I find the syntax unfriendly, and for the most complex cases, I prefer parsing the JSON output with jq. Anyway:

Formatting expect commands

sho$ CID=$(sudo docker run --rm -d busybox sleep 3000)
sh$ sudo docker inspect $CID
[
    {
        "Id": "0470a3549d2f41efd0477cf8fc4eb3a27aca592f089c018ad24704031bbef967",
        "Created": "2019-02-05T16:41:40.040349047Z",
        "Path": "sleep",
        "Args": [
            "3000"
        ],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 26467,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2019-02-05T16:41:40.659056612Z",
            "FinishedAt": "0001-01-01T00:00:00Z"
        },
        "Image": "sha256:3a093384ac306cbac30b67f1585e12b30ab1a899374dabc3170b9bca246f1444",
        "ResolvConfPath": "/var/lib/docker/containers/0470a3549d2f41efd0477cf8fc4eb3a27aca592f089c018ad24704031bbef967/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/0470a3549d2f41efd0477cf8fc4eb3a27aca592f089c018ad24704031bbef967/hostname",
        "HostsPath": "/var/lib/docker/containers/0470a3549d2f41efd0477cf8fc4eb3a27aca592f089c018ad24704031bbef967/hosts",
        "LogPath": "/var/lib/docker/containers/0470a3549d2f41efd0477cf8fc4eb3a27aca592f089c018ad24704031bbef967/0470a3549d2f41efd0477cf8fc4eb3a27aca592f089c018ad24704031bbef967-json.log",
        "Name": "/vibrant_shaw",
        "RestartCount": 0,
        "Driver": "aufs",
        "Platform": "linux",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "docker-default",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": null,
            "ContainerIDFile": "",
           "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "default",
            "PortBindings": {},
            "RestartPolicy": {
                "Name": "no",
                "MaximumRetryCount": 0
            },
            "AutoRemove": true,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": null,
            "CapDrop": null,
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "shareable",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": [],
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": [],
            "DeviceCgroupRules": null,
            "DiskQuota": 0,
            "KernelMemory": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": false,
            "PidsLimit": 0,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "MaskedPaths": [
                "/proc/acpi",
                "/proc/kcore",
                "/proc/keys",
                "/proc/latency_stats",
                "/proc/timer_list",
                "/proc/timer_stats",
                "/proc/sched_debug",
                "/proc/scsi",
                "/sys/firmware"
            ],
            "ReadonlyPaths": [
                "/proc/asound",
                "/proc/bus",
                "/proc/fs",
                "/proc/irq",
                "/proc/sys",
                "/proc/sysrq-trigger"
            ]
        },
        "GraphDriver": {
            "Data": null,
            "Name": "aufs"
        },
        "Mounts": [],
        "Config": {
            "Hostname": "0470a3549d2f",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": [
                "sleep",
                "3000"
            ],
            "Image": "busybox",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": {}
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "654da638965d9bb26e394ba61036ea713bdccb6304eb09bee86f3c0611a26ca5",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {},
            "SandboxKey": "/var/run/docker/netns/654da638965d",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "c69ef9cb870a3c477105c7bc19e370683886e4e3c62fac8444d3e1e097551250",
            "Gateway": "172.17.0.1",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "172.17.0.3",
            "IPPrefixLen": 16,
            "IPv6Gateway": "",
            "MacAddress": "02:42:ac:11:00:03",
            "Networks": {
                "bridge": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "5ec8728b05efd5ce6bfde5cbe6e42f247348345e0fdfb305fda76566327fc151",
                    "EndpointID": "c69ef9cb870a3c477105c7bc19e370683886e4e3c62fac8444d3e1e097551250",
                    "Gateway": "172.17.0.1",
                    "IPAddress": "172.17.0.3",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:11:00:03",
                    "DriverOpts": null
                }
            }
        }
    }
]

Returning scalar properties

You can query any scalar attribute by its name, prefixed by a dot:

sh$ sudo docker inspect $CID --format '{{.Id}}
0470a3549d2f41efd0477cf8fc4eb3a27aca592f089c018ad24704031bbef967
sh$ sudo docker inspect $CID --format '{{.Name}} {{.Driver}}'
/vibrant_shaw aufs

# jq notation
sh$ sudo docker inspect $CID | jq '.[].Id'
"0470a3549d2f41efd0477cf8fc4eb3a27aca592f089c018ad24704031bbef967"

sh$ sudo docker inspect $CID | jq '.[]|(.Name,.Driver)'
"/vibrant_shaw"
"aufs"

You can also query nested fields:

sh$ sudo docker inspect $CID --format '{{.State.Running}}'
true

# jq notation
sudo docker inspect $CID | jq '.[].State.Running'
true

Returning entire objects and arrays

But it does not work as expected for nested structures:

sh$ sudo docker inspect $CID --format '{{.State}}'
{running true false false false false 26467 0  2019-02-05T16:41:40.659056612Z 0001-01-01T00:00:00Z <nil>}

You end up having the data, but without the field names, it is pretty useless--unless you are assuming some predefined field order (something that is probably not safe).

When accessing entire objects, there are chances you want an explicit formatter:

sh$ sudo docker inspect $CID --format '{{json .State}}'
{"Status":"running","Running":true,"Paused":false,"Restarting":false,"OOMKilled":false,"Dead":false,"Pid":26467,"ExitCode":0,"Error":"","StartedAt":"2019-02-05T16:41:40.659056612Z","FinishedAt":"0001-01-01T00:00:00Z"}

# jq notation
sh$ sudo docker inspect $CID | jq '.[].State'
{
  "Status": "running",
  "Running": true,
  "Paused": false,
  "Restarting": false,
  "OOMKilled": false,
  "Dead": false,
  "Pid": 26467,
  "ExitCode": 0,
  "Error": "",
  "StartedAt": "2019-02-05T16:41:40.659056612Z",
  "FinishedAt": "0001-01-01T00:00:00Z"
}

Same thing for arrays:

sh$ sudo docker inspect $CID --format '{{.HostConfig.ReadonlyPaths}}'
[/proc/asound /proc/bus /proc/fs /proc/irq /proc/sys /proc/sysrq-trigger]

sh$ sudo docker inspect $CID --format '{{json .HostConfig.ReadonlyPaths}}'
["/proc/asound","/proc/bus","/proc/fs","/proc/irq","/proc/sys","/proc/sysrq-trigger"]

sh$ sudo docker inspect $CID --format '{{join .HostConfig.ReadonlyPaths ":"}}'
/proc/asound:/proc/bus:/proc/fs:/proc/irq:/proc/sys:/proc/sysrq-trigger

# jq notation
sh$ sudo docker inspect $CID | jq '.[].HostConfig.ReadonlyPaths'
[
  "/proc/asound",
  "/proc/bus",
  "/proc/fs",
  "/proc/irq",
  "/proc/sys",
  "/proc/sysrq-trigger"
]
sh$ sudo docker inspect $CID | jq '.[].HostConfig.ReadonlyPaths | join(":")'
"/proc/asound:/proc/bus:/proc/fs:/proc/irq:/proc/sys:/proc/sysrq-trigger"

Arrays of objects

More complicated (according to my limited Go Template skills) is accessing fields nested in an array of objects:

# This works as expected
sh$ sudo docker network inspect bridge --format '{{json .IPAM.Config}}'
[{"Subnet":"172.17.0.0/16","Gateway":"172.17.0.1"}]

# But not that
sudo docker network inspect bridge --format '{{.IPAM.Config.Gateway}}'

Template parsing error: template: :1:7: executing "" at <.IPAM.Config.Gateway>: can't evaluate field Gateway in type interface {}

When I want to extract such nested value, I resort on an explicit range loop:

sh$ sudo docker network inspect bridge --format '{{range .IPAM.Config}}{{.Gateway}}{{end}}'
172.17.0.1

# jq notation
sh$ sudo docker network inspect bridge | jq '.[].IPAM.Config[].Gateway'
"172.17.0.1"

sh$ sudo docker network inspect bridge | jq -r '.[].IPAM.Config[].Gateway'
172.17.0.1

Formatting ls commands

With the json formatter, we can obtain (some? all?) the possible selectors you can later use with --format:

sh$ sudo docker network ls --format '{{json .}}'
{"CreatedAt":"2019-01-07 14:28:55.63238018 +0100 CET","Driver":"bridge","ID":"5ec8728b05ef","IPv6":"false","Internal":"false","Labels":"","Name":"bridge","Scope":"local"}
{"CreatedAt":"2019-02-05 15:36:57.701603791 +0100 CET","Driver":"bridge","ID":"45d6f09ca923","IPv6":"false","Internal":"false","Labels":"com.docker.compose.network=default,com.docker.compose.project=compose-demo,com.docker.compose.version=1.23.2","Name":"compose-demo_default","Scope":"local"}
{"CreatedAt":"2019-02-05 17:30:47.98883918 +0100 CET","Driver":"bridge","ID":"aaff418d1c56","IPv6":"false","Internal":"false","Labels":"com.docker.compose.network=net,com.docker.compose.project=compose-demo,com.docker.compose.version=1.23.2","Name":"compose-demo_net","Scope":"local"}
{"CreatedAt":"2018-05-26 10:31:48.149819089 +0200 CEST","Driver":"host","ID":"988954899b53","IPv6":"false","Internal":"false","Labels":"","Name":"host","Scope":"local"}
{"CreatedAt":"2018-05-26 10:31:48.118573602 +0200 CEST","Driver":"null","ID":"ce1680ed377f","IPv6":"false","Internal":"false","Labels":"","Name":"none","Scope":"local"}
{"CreatedAt":"2019-02-05 19:44:21.286955573 +0100 CET","Driver":"bridge","ID":"41c62c023d16","IPv6":"false","Internal":"false","Labels":"com.docker.compose.version=1.23.2,com.docker.compose.network=default,com.docker.compose.project=test","Name":"test_default","Scope":"local"}

Here, you can see docker network ls exposes, among others, the "ID", "Driver" and "Scope" fields. You can use them in your format:

sh$ $ sudo docker network ls --format '{{.ID}}: {{.Driver}} ({{.Scope}})'
5ec8728b05ef: bridge (local)
45d6f09ca923: bridge (local)
aaff418d1c56: bridge (local)
988954899b53: host (local)
ce1680ed377f: null (local)
41c62c023d16: bridge (local)

The best of both worlds

The docker ... ls commands only exports a small subset of the docker object's properties. Depending your needs, you may prefer something along the lines of docker ... inspect $(docker ... ls -q).

For example, docker image ls does not export the image architecture. but you can obtain it with docker image inspect. If you want that information, you could use something like that:

sh$ sudo docker image inspect \
         --format '{{.Id}}: {{.RepoTags}} ({{.Architecture}})' \
         $(sudo docker image ls -q)
sha256:fc37804350a8adea0b41dba0c413a6c967140ca2ed6dfeefdbe88d7868902406: [compose-demo_monitor:latest] (amd64)
sha256:a6f30c0890bca94e3cf5930ad5c3358b06cb38450e92d4ddabeb208ee34383f8: [compose-demo_app:latest] (amd64)
sha256:a79892b33e0bb51598e9311cc1d23ca8d0e3013a5a15fd67d202d22ffd08a6af: [] (amd64)
sha256:394c82635f220db389b8532e316001674965b51679dbc8dff747d5f224e6d568: [] (amd64)
sha256:95f0c755feb65af441c0cb8727bb13b35517977a3bed06e4a3456c95d6a69dd4: [redis:5.0-alpine] (amd64)
sha256:1f6c34f7921c7b50ef4452737399b5cff0eb834fc183d188d82886fd23b7fc34: [node:8] (amd64)
sha256:3a093384ac306cbac30b67f1585e12b30ab1a899374dabc3170b9bca246f1444: [busybox:latest] (amd64)


Which Release Management aspects help explain the difference between Waterfall and Agile?

IMO DevOps is culture, much like Agile (without choosing an agile methodology.) Therefore you don't "do" DevOps.
You "do" a release methodology called Continuous Delivery as part of a DevOps Culture. (full disclosure, I don't think I've ever referred to CD as a release methodology before, but in my jetlagged state I think it works)
If you'll buy that, then here's the definition of Continuous Delivery from one of the people who wrote the book by the same title, Jez Humble.

Continuous Delivery is the ability to get changes of all types—including new features, configuration changes, bug fixes and experiments—into production, or into the hands of users, safely and quickly in a sustainable way.
Our goal is to make deployments—whether of a large-scale distributed system, a complex production environment, an embedded system, or an app—predictable, routine affairs that can be performed on demand.
We achieve all this by ensuring our code is always in a deployable state, even in the face of teams of thousands of developers making changes on a daily basis. We thus completely eliminate the integration, testing and hardening phases that traditionally followed “dev complete”, as well as code freezes.

So then, you can work in an Agile methodology, having software you can demonstrate to the business, making sure you're doing proper automated testing, reacting well the change and all of the things that make it better than waterfall. All too often that doesn't mean you could actually deploy it to production.
You end up with something like this:

So the software will (probably) be better when you're done then if you didn't have some sort of iterative approach, but you don't really know because real users haven't ever seen it.
What you really want is something that looks more like this:

Every iteration, something gets deployed to production. So, the software is deployed. If you decide to create downloads, open up the web server, or however you get software into the hands of users it's released.
What the heck!? I asked about DevOps! Nobody asked about Continuous Delivery!!
So what does DevOps have to do with this?
It's very, very hard (approaching impossible) to truly have your software in a state where you could deploy it whenever you want unless that team is working in a DevOps culture. A culture where System Admins, DBA, SREs, Security People, Devs, QAs, etc., etc., are all part of a single team and not siloed part of an organization with handoffs.
Note:
About part of a comment posted to this answer, which was like so:

About your "... software in a state where you could deploy it whenever you want ...": that reminds me about "automatic pilot" software (in a plane) ... My favorite question about that: "Imagine an update is applied to such software ... How would you feel about doing so inflight ... While you're in onboard?".

I love that question (in bold, in the above quote)! The idea of "is it really ready?" is something I rant about all the time - blog. IMO it's vital that you're confident in security, performance and other too often "secondary" tests in order to practice CD. The features are done when they're done, but hackers are always there.


I will make my answer, based on the knowledge for security testing, but IMHO this can be generalized.

Black box testing - when the tester know nothing about the system, components, liaisons, connections, etc. This can be helpful more like UI/UX testing, functional testing. Example: you do not work for Microsoft and also you do not have the source code and need to test MS Office
Gray box testing - when the tester have some knowledge about how the system work, some information about internals.
White box testing - when the tester have or can get all the information about internals, connections, principles of work of the tested product. Example: testing Linux kernel.

For reference:

Black box testing (Wikipedia)

White box testing (Wikipedia)

Gray box testing (Wikipedia)


How to get the device name of a specified mount point with ansible facts?

Use json_query. For example
    - set_fact:
        my_device: "{{ ansible_mounts|json_query(query) }}"
      vars:
        query: "[?mount=='/'].device "
    - debug:
        var: my_device

gives
    my_device:
      - /dev/sdc1

To get the first element of the list only append the filter first
    - set_fact:
        my_device: "{{ ansible_mounts|json_query(query)|first }}"


If run you docker pull <image>, you are implicitly asking for a tag named latest.  There is no such tag available from the Ansible repository.  You can see a list of available tags at https://hub.docker.com/r/ansible/ansible/tags/.

For example, if you wanted the fedora27py3 version of the image, you would run:

docker pull ansible/ansible:fedora27py3

I note that the most recent of thoses images is 7 months old. If you're experimenting iwth Ansible you might want to just install it into a Python virtual environment.


Key attributes of Continuous Integration (CI) software

In general, there are five main differences between different CI software solutions.

Cost: Is the software open source and free or proprietary? Does your DevOps team already have a budget for software or are you expected to compare free options?
Maintenance: Is the software something you need to host on your own and maintain, or is it being offered as a service? If you need to host it on your own, do you have the team members available who can run the maintenance? If it is being provided as a service, where does your code reside?
Security: Are there any security or compliance concerns about where the build machines are hosted? Are the permissions required by the CI software into your SCM well understood?
Community: Does the CI software have a good community around it with the plugins and support for the technologies you use? Is it easy to get support or troubleshoot when things go wrong?
UX: How easy is the software to use? Does the CI software support scripts stored within your SCM for the tasks you need to do? Is the reporting easy to understand or will it require some training?


Unable to get jupyter/scipy-notebook running on docker

It turned out to be an issue with port mapping. After mapping local machine port 8888 to docker's 8888 I could launch the program in browser.

sudo docker run -p 8888:8888 jupyter/scipy-notebook


Elasticsearch on Docker - file permission issues

I have discovered the issue, the path to the role_mapping.yml file was incorrect in elaticsearch.yml. I had to use the full path /usr/share/elasticsearch/config/role_mapping.yml


Adding a persistent volume to a Kubernetes deployment from Gitlab Auto Dev-ops

The answer is that all Auto Dev-Ops deployments at present use Helm to package and deploy your application.

In order to make changes to the deployment, such as adding a persistent volume claim, you should download and modify the Helm chart that GitLab uses.

The files in this repo should go into a new directory chart/, or you can link to another location by setting the AUTO_DEVOPS_CHART project variable to a URL.

This page has more details.


Permission errors with non-root USER when running on Linux

As described in https://github.com/docker-library/tomcat/issues/35, this is related to the storage driver used (aufs).  Changing to use overlay2 solved the issue:

https://docs.docker.com/storage/storagedriver/overlayfs-driver/


Can we consider usage of git lfs for artifacts equally mature strategy to an artifactory?

What is Git-LFS?

Git Large File Storage (LFS) prevents that large files need to be stored in git itself.

https://git-lfs.github.com/

  Git Large File Storage (LFS) replaces large files such as audio
  samples, videos, datasets, and graphics with text pointers inside Git,
  while storing the file contents on a remote server like GitHub.com or
  GitHub Enterprise.

What is artifactory?

Artifactory stores artifacts, e.g. docker images, jars, wars

Artifactory vs. Git-lfs

Artifactory is used to store artifacts while git-lfs is used to prevent that large files are stored in git. If one will store large files in git and one would like to clone this repository it will take a decade before it is cloned.


Why is the AWS EC2's spot price greater than the on-demand price?

This is actually a great example of people slightly abusing spot. People are saying 'Our workload is really important but we don't want to pay full on demand price', so they set a bid price higher than on-demand on the assumption that it is very unlikely to be terminated, but still want to get the 'cheapest possible' spot price on offer.

There have been cases where people enter, for example, $1000 (I've been told of at least one time this happened) because they want the benefit of the spot market. Of course naturally at some point the demand comes in and the spot price SOARS to make people pay higher than on-demand.

The way the spot market works is that Amazon have X instances spare capacity, and they count from the top down until they fill the need for all X instances. The 'price', then, is the lowest price at which they can fulfill those X instances.

So imagine Amazon have 10,000 instances - well they will count down to (say) $0.43 until they've got those 10,000 instances fulfilled. But if that supply suddenly drops to 100 instances, then maybe a few people put bid prices of $10,000 for their 100 instances, suddenly they'll be paying that $10k per hour.

Tl;dr Understand how spot works, and set a cap you are prepared to pay.


Workflow with docker in the development environment

We use kubernetes / minikube rather than docker-compose, but its the same problem.
We generally go with option #1 as:

Development is 99% getting the app right.
You want to be able to iterate fast/freely with your app.
You want easy use of your debugger / IDE without inhibition.
It lets you avoid configuring / managing remote debuggers from within the docker container.

If you did this in the container, you'd have to worry about making sure you didn't actually put it in prod as it can be harmful; see below.

For most code changes, we actually just have GitLab-CI set up so that it test-builds/deploys the container/app on check in (on their own branch/space).  That way, most developers don't even need to think about the container part unless they're making mods to that area.  But, obviously you'd still want them being able to run the MySQL container locally and easily.

Remote Debugging Info

https://jaxenter.com/remote-debugging-java-applications-151466.html

  Warning: Don’t keep remote debugging JVM arguments ON always, as it has following downsides:

  Remote debugging mode disables several optimizations that JVM does to application to optimize the performance. All those optimizations will be lost.
  Remote debugging opens up a port. It’s a security risk, as anyone who can hit the server can initiate remote debugging


Yes, you are interpreting that correctly.
When you express your SLO as a percentage, you should express your error rate as a percentage too. So, in you example your error rate of 1 is 100%. That would make the equation:

(100-95/100) * 1 hour * 1 = 0.05 hours = 3 minutes

The period in that equation is the reporting period that you have chosen for your SLO. You have not mentioned what time period you chose. The examples in the workbook are 30 days. If you use a 30 day (720 hour) period in your example, you would consume approximately 0.14% of your error budget in the 3 minutes it takes for the alert to fire:

( 1 * 1 hour ) / 720 hours = 0.14%

The rates are different because they are measured over different time periods. (Note that the graph is using a logarithmic scale, so the actual error rate and the peak of the 5m measurement is 15%, not 10%.)

Consider what you would measure one minute after the errors started:

The 5m measurement would see 4 minutes of no errors plus one minute of 15% errors, which gives an overall error rate of 3%.
The 60m measurement would see 59 minutes of no errors plus one minute of 15% errors, which is just an overall error rate of 0.25%.

Then 5 minutes after the errors started:

The 5m measurement would see 5 minutes of 15% errors, so the overall rate would be 15%.
The 60m measurement would see 55 minutes of no errors plus 5 minutes of 15% errors, so the overall rate would be 1.25%. This is actually still too low to trigger the alert, so it will actually take more than 5 minutes to alert.

Using the equation from before, the actual time taken to alert is 5.76 minutes:

5.76 = (0.001/0.15) * 60 * 14.4

Your PromQL seems to be looking at the good events instead of the bad events, as the le="2" selector is for the bucket counting latencies less than or equal to (le) 2 seconds. You may want to create a separate "good_latency_rate1h" recording rule for the good events, and then use something like "(1-good_latency_rate1h) > (14.4*0.001)" for your alerts.


What are the security risks of CD in an open source project?

You essentially have three threats here:

Someone could modify .travis-ci and use that to exfiltrate the unencrypted key material.
You could accidentally check-in or publish the decrypted key material as part of your release process.
Someone could attempt to attack the encrypted key-material directly; this is very unlikely given modern practice and current computing power.

The challenge of secret management is common in open source projects; one approach I have taken in the past is to have a second "deploy" stage in a private repository and pipeline.  A private repository and pipeline, then ring-fences the sensitive data from malicious actors entirely, protecting yourself from 1 and 3.  You can safeguard against two by whitelisting files that can be checked in to decrease the probability of making a mistake.

Finally, an approach that can be used to increase the detectability of an attack is to store the key in AWS CloudHSM, Google Cloud HSM or Azure KeyVault with the goal of marking the private key as non-exportable and then handing off any encrypt, wrap or sign requests to the cloud.  This means even if the credentials for the HSM/KeyVault do get exfiltrated you know every time they are used to sign something and can effectively revoke access to the key entirely.  Unfortunately, there doesn't seem to be any way of doing this with SSH keys at the moment, however, it works will with code-signing certificates.


For the best practices will be great to install Jenkins master on the lower Environments (best Dev env) and to leave Production in peace with only necessary things (with your end-user application and their dependencies).

The reason why is because production means production and if something will happen with Jenkins or if you will allow a lot of people accessing Jenkins in production you increase your failure rate.

Will be better to install Jenkins separated from production and to do VPC Peering between clusters and after you can create a fully CI/CD pipeline where you can create a Jenkins slave/agent which will deploy your application to PROD.

PROD suppose to be more focused to an end Product result.
You didn't specify which cloud provider do you use so, I can't help too much...
Hope this will clarify your vision.


Do I need an orchestration tool to autoscale/scale on demand my web service?

Yes. And there are tools out there covering this piece of functionality.

Probably the most popular one is kubernetes (see also kubernetes):

  Kubernetes (k8s) is an open-source system for automating
  deployment, scaling, and management of containerized applications.

Depending on the provider for your deployment environment the actual orchestration tool/service may be already included and all you'd provide would be your desired scalability configuration. For example when deploying on the Google App Engine Flexible environment the service scaling settings could look like this:

automatic_scaling:
  min_num_instances: 1
  max_num_instances: 15
  cool_down_period_sec: 180
  cpu_utilization:
    target_utilization: 0.6


The two approaches whilst somewhat overlapping solve two different problems:

Feature flags allow you to decouple the action of deployment from the action of release.  i.e. you can push a change to an API that adds functionality to support a new feature, then when your product owner wants to release it all they need to do is toggle the feature on.
API versioning enables you to make changes to your API without breaking compatibility with older clients.

In both cases, you need to ensure that old feature flags and old API versions are eventually removed from the codebase or it will become very complicated very quickly.

The deciding factor comes down to whether or not you have control over all of your clients if you can guarantee that you can make changes to both your clients and the API in such a way that they will remain compatible, or incompatible clients can be removed from your ecosystem very quickly you probably don't need API Versioning.


Is it necessary to specify resource in S3 Bucket Policies?

Yes, Resource is a required element on S3 bucket policies. If the resource is omitted, the API will respond with an error resembling this one below.

status code: 400
Error putting S3 policy: MalformedPolicy: Statement is missing required element

The reasoning is that each statement could apply to the bucket itself, the contents of the bucket, or some subset of the bucket contents, and it's up to the user to clarify. In theory AWS could provide a default that includes both the bucket and all of its contents, but that design probably wasn't chosen because it makes it too easy to accidentally make the bucket policies broader than expected.

Additionally, it is not permissible to use a wildcard in the resource element of an S3 bucket policy. Attempts to do so will yield an error such as this:

Error putting S3 policy: MalformedPolicy: Policy has invalid resource


Run integration test with ansible playbook using terraform provisioning

You are right about Terraform having a plan-apply cycle which may not be that conducive to automation. The following has worked for me in similar situations:

Use Terraform to "develop" new versions of a base EC2 image.
Use Ansible playbooks to provision EC2 instances off a specific version of the image and run tests on them.


How to set CloudWatch alarms for non-compliancy in AWS Config rules?

From what I know of AWS Config there's no way at the moment to directly publish a metric from which you can base an alarm.

If the notification has to come from a Cloudwatch alarm then AWS Config will let you execute Lambda based on rules set up in AWS Config - link.

If you want a simpler option AWS Config also lets you publish directly to an SNS topic when your rule is evaluated - link

Failing this you can also watch for CloudTrail events and then publish a CloudWatch metric which you can base an alarm on. This is fairly in depth though and if the important part is getting the emails to be looked at by a person then your best bet is probably to have AWS Config publish directly to SNS itself.


Kubernetes auto scaling in cloud environment

Recently Kubernetes started preview work on https://cluster-api.sigs.k8s.io which is meant to be the standard way to solve the way how High Availability clusters are deployed.

The kubeadm tool has also been updated with features to deploy multiple masters since this question was first asked, which help tremendously. With the exception of deploying cloud-specific configuration, the documentation for kubeadm configuration YAML file brings you quite the way towards having a properly deployed cluster.

docs for kubeadm: https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm/


There are some fundamental misunderstandings in your bullets which makes your question unclear:

Apple will not allow you to develop/build using a virtual system.  I believe this is a legal constraint.  That said, you can run pretty much anything in VirtualBox as long as you have a legally provided image (e.g. a Linux image).
You can only run the Windows base images if you are running Windows Containers which will be incompatible with MacOS systems.  If you want to build on MacOS using Docker you will need to use Linux containers.

However, you can run your continuous-integration runners across Windows and Mac.  This is one of the major benefits of implementing runners in Docker for your CI tool.  Now if you have agents on Windows, MacOS, or even Linux, you can get a consistent build experience by using Docker. This official GitLab article has more about setting this up.


What is the performance different running docker on Windows versus Linux?

The performance difference between linux containers running on Windows, Linux, or Mac will depend entirely upon how much ram you allocate to the Docker Daemon.  The difference should not be noticeable.  Additionally, you will be able to take advantage of the Alpine base image (~5 Mb) for small, compact containers.

If you need to run IIS in your container, then you should look at windows containers.  The disadvantage is that the containers will be significantly larger in size, and you will lose cross-platform compatibility.  Docker windows containers.