The Reddit thread asks:

  Why is the Docker Mind Shift so hard?

The OP then links to The Docker Mind Shift and states that

  The Docker Mind Shift helps explain the challenges many people struggle with, in DevOps and containers. Thoughts?

(Currently,) the highest upvoted comment states

  to be honest my biggest challenges have been with the speed of tooling changes rather than the concepts. In theory, i get the workflow and ideas behind docker fairly well, but in practice sometimes it does feel like you're throwing a whole load of plates up in the air and then walking off before they hit the ground again. I get that as a new tech idea it moves fast, but just trying to keep up with the ever moving goalposts can be exhausting. It does seem to have gotten better in the last 2 years though to be fair.

In response to this comment, sirex007 responds

  after finishing any reasonably sized project in docker, i'm always reminded of this image

and links to the above image. This image is attempting to comically illustrate the confusion many feel with the world of DevOps. There is a smorgasboard of toolkits and frameworks out there, and the bullets in the image are a commentary of the nonsense of the diagram to the right.

This is seen in the first bullet:

  Did you just pick things at random?

The subsequent bullets then expand on this question. For example, Redis is a caching system and stores key-value pairs. Similarly, MongoDB is a type of NoSQL database which also supports key/value pairs and caching, so it is a little silly to have both Redis and MongoDB.

It seems that sirex007 feels that this graphic illustrates the point that I get that as a new tech idea it moves fast, but just trying to keep up with the ever moving goalposts can be exhausting. and my biggest challenges have been with the speed of tooling changes as there are a lot of trendy tools appearing in the graphic, which is making the point that the implementer doesn't seem to understand the toolkits in use in his or her stack well.

This is a fair point: with the DevOps goal of rapid iteration and deployments, the appearance of new tools on a frequent basis and the changes within those tools could certainly seem like they are moving at a breakneck pace. I would argue that this is evidence of the success of the DevOps model however. If these projects are able to successfully iterate rapidly using DevOps, it is proof of the model's successes - though perhaps the cost is trying to keep up with that rapid pace.


Google CloudPlatform remove/delete app deployed during tutorial steps

Yes, disabling an application prevents charges, but only from the App Engine prospective, there could still be other changes from other Cloud products usage. From Disabling an application:

  Your app's resources in both the standard and flexible environment are
  disabled and therefore, won't incur charges. However, billing charges
  can still occur for the other services in your Cloud Platform project,
  for example you can still incur storage charges for your stored data.

To completely end all Cloud billing for a project you can shut it down (i.e. mark it for deletion!). But please note that you won't be able to re-use its ID, ever.


Does Immutable Servers imply no source code updates on the servers?

No.

Immutable is exactly what it means, immutable, no change on configuration or code running or system library or whatever, if a change has to be made, create a new image and deploy it, never change it while running.

Source code updates are the least thing to change on a running server, this should not happen on a running server, immutable or not.

You should create a versioned release of your code to deploy on servers, once you have this level of fixed versioning you can step ahead to versioning your server image which include this code release.

Any change, being a code patch or a system patch or a configuration change should trigger a new release to be deployed.


Node-centric network metrics are very different from the sum of metrics of the Pods running on that node

I found out what was happening in my graphs. All the Pods mentioned above have one thing in common: they use the host's network namespace, so their network metrics are all identical, and equal to the global host's metric (just with a slightly different precision).
$ kubectl - monitoring get pod -o jsonpath='{.spec.hostNetwork}' \
    prometheus-stack-prometheus-node-exporter-jnhw7

true

$ kubectl -n kube-system get pod -o jsonpath='{.items[*].spec.hostNetwork}' \
    kube-proxy-gke-triggermesh-product-control-plane-7fc0ad24-z586 \
    gke-metrics-agent-5cv4m \
    prometheus-to-sd-tk8jv \
    fluentbit-gke-xh879

true true true true

One way to see it is to compare the host's metric to one of the above Pods:


Is there a book only about Ansible (without touching Vagrant, VirtualBox, Docker and virtualization stuff)?

I would highly recommend looking at this tutorial by digital ocean. How to install and configure Ansible on Ubuntu.

Although this tutorial is not a book, I had a similar situation with the ansible docs and trying to understand them when I first got started. This tutorial will provide an in depth hands on experience for getting started with ansible.


Looking for a Desired State and Configuration as Code central configuration management framework

Ansible is one example of a tool that that allows you to define all your configuration as scripts stored in your SCM and can deal with configuration drift.  Ansible has modules to work with Windows in multiple ways including DSC through run books.  Full information is available at:  https://docs.ansible.com/ansible/latest/user_guide/windows.html.

For desired state configuration specifically - https://docs.ansible.com/ansible/latest/user_guide/windows_dsc.html#what-is-desired-state-configuration

There is also additional information in writing scripts to insure the system stays in the desired state.  It is important to insure the way your writing the script does not in itself make differences in the system. - https://shadow-soft.com/ansible-idempotency-configuration-drift/

I know of many environments where the full system is defined and deployed with Ansible and it is used to remove any drift.

There are other ways to also help with the situation including using dynamic provisioning to shorten the life of systems.  Only keeping them up for the length of a test or other short time frame.


How to measure Lead Time for a Scrum Team

The idea of lead time is that when the customer asks for something, a clock starts ticking in their head and that clock stops when they have it in their hands. Therefor, a feature's lead time would be from when the request is received (this usually corresponds to adding it to the backlog) until it is delivered and usable to the requestor.

Full disclosure, I haven't read Accelerate, but since Forsgren is using an existing term and it seems it's being used correctly, I am referring to the proper definition of the term.

Now, that isn't to say all those other things you mention aren't valuable. They may be. They may even be more valuable to you than lead time. They just aren't lead time.


'[[0;1;31mFAILED[0m] Failed to start' when running cloud-init?

From my rather limited experience, a likely reason is that the command sudo docker image build does not find the docker file.
(But a virtual machine may have been successfully created.)


How can we get the jenkins console output in a text file?

if you want just access the log and download it as a txt file to your workspace from the job's URL:

${BUILD_URL}/consoleText

On Linux, you can use wget to download it to your workspace

wget ${BUILD_URL}/consoleText

The actual log file on the file system is in the Master machine. You can find it under:

$JENKINS_HOME/jobs/$JOB_NAME/builds/lastSuccessfulBuild/log


If you are unsure what happened, try deploying it in AWS via AWS vault quickstarts. They deploy vault end-to-end, so now you can compare it to yours and find out what was the problem.


Cancel sibling snapshot dependency on failure

I was unable to find a way to configure the build to behave like I wanted. Instead I needed to start using the TeamCity API to achieve this.

In the build.sh and test.sh file that TC runs, I extended the failure function (simple bash function we call whenever our build script fails):

teamcityFailure () {
    . ./cancel_build_chain.sh
    echo "##teamcity[buildProblem description='$1']"
    exit 1
}

This now calls off to this new cancel_build_chain.sh script, which looks something like this:

#!/bin/bash

teamcityProgress () {
    echo "##teamcity[progressMessage '$1']"
}

teamcityProgress "Build step has failed, attempting to cancel the rest of the build chain..."

BUILD_ID=$TEAMCITY_BUILD_ID # passed in from build chain configuration

# Use the current failed step ID to query for the snapshotDependency "from:" here - which returns the final "docker publish" step
FINAL_STEP_DATA=$(curl --silent --request GET \
    "https://teamcity/app/rest/builds/project:MyProjectName,snapshotDependency:(from:(id:$BUILD_ID),includeInitial:true),defaultFilter:false" \
    --header "Content-Type: application/xml"")

# From the "docker publish" step, parse the XML and fetch out all dependent step IDs. One will be this current step, the others will be its sibling steps
# We do this twice, once for queued steps, and once for currently running steps. This is because we need to cancel running vs queued builds differently.
QUEUED_DEPENDENT_STEP_IDS=$(echo $FINAL_STEP_DATA \
 | grep -Eo '<snapshot-dependencies.+<build id="[0-9]+".+state="queued"' \
 | grep -Eo 'id="([0-9]+)"' \
 | grep -Eo '[0-9]+')

# Now remove the current step ID
QUEUED_DEPENDENT_STEP_IDS=${QUEUED_DEPENDENT_STEP_IDS/$BUILD_ID/}

# trim leading / trailing whitespace
QUEUED_DEPENDENT_STEP_IDS=QUEUED_DEPENDENT_STEP_IDS | sed 's/ *$//'

RUNNING_DEPENDENT_STEP_IDS=$(echo $FINAL_STEP_DATA \
 | grep -Eo '<snapshot-dependencies.+<build id="[0-9]+".+state="running"' \
 | grep -Eo 'id="([0-9]+)"' \
 | grep -Eo '[0-9]+')

# Now remove the current step ID
RUNNING_DEPENDENT_STEP_IDS=${RUNNING_DEPENDENT_STEP_IDS/$BUILD_ID/}

# trim leading / trailing whitespace
RUNNING_DEPENDENT_STEP_IDS=RUNNING_DEPENDENT_STEP_IDS | sed 's/ *$//'

for DEP_ID in $QUEUED_DEPENDENT_STEP_IDS
do
    teamcityProgress "Cancelling queued build step $DEP_ID"
    curl --silent --request POST \
        "https://teamcity/app/rest/buildQueue/project:MyProjectName,id:$DEP_ID" \
        --data "<buildCancelRequest comment='Another part of the build chain failed so this sibling step was cancelled.' readdIntoQueue='false' />" \
        --header "Content-Type: application/xml"
done

for DEP_ID in $RUNNING_DEPENDENT_STEP_IDS
do
    teamcityProgress "Cancelling running build step $DEP_ID"
    curl --silent --request POST \
        "https://teamcity/app/rest/builds/project:MyProjectName,id:$DEP_ID" \
        --data "<buildCancelRequest comment='Another part of the build chain failed so this sibling step was cancelled.' readdIntoQueue='false' />" \
        --header "Content-Type: application/xml"
done

Not super elegant but it gets the job done and prevents build agents sitting there running build steps for builds which have already failed.


Is there a php (or composer) command to change upload_max_filesize && post_max_size from terminal?

The reason; having a more a version-agnostic and more
  backward-compatible directive in my server-environment-establishment
  script, to change the values of these two variables.

If you're trying to configure a server, you should use a configuration management system (Ansible, Puppet, Chef, Salt) or bake images for immutable infrastructure; trying to hack in pieces of configuration via PHP or bash or whatever will just make you unhappy.


Suggestions for a good run command review console

I am looking for a way for our members to submit scripts without fussing too much about format or folder structure.

It sounds like you need some combination of testing and linting, perhaps with a few utility scripts to organise and gate content from landing in the repo.
The "Vetting process" you mention sounds to me exactly like "tests need to pass on the scripts". A widely-used framework to write such tests is the Bash Automated Testing System - BATS.
It would be up to you to write the tests to check whether, as you desire, these scripts are "secure, compliant and won't create an issue", but at least you have a framework for generating and running them.
At a more superficial level, you could use the Shellcheck linter to assert that any proposed new code adheres to whatever set of common rules you adopt in your community.
Pulling these together into a GitHub action (e.g. using https://github.com/github/super-linter), or a git pre-commit hook etc could be a good way to add a quick inspection to new contributions, and help maintainers.
These would provide quantitative measures of correctness. In order to also provide some qualitative or more subjective measures of acceptance, this would depend greatly on the operating environment and which tools would fit naturally into yours.
However, some tools do attempt to address this need by design. Without further knowledge of the environment it's difficult to provide concrete answers, but here is a comparison of two options:
GitHub, GitLab requests
Assuming the code is destined for a GitHub or GitLab repo, GitHub or GitLab itself could of course be used as the "review console". A contributor would fork the repo, send a new contribution in the form of a PR against the repo, and then the checks referred to above would be executed.
The actions could take care of the organisation, based on some internal convention, e.g. using labels, i.e. "file scripts in PRs labelled admin in the admin directory".
Gerrit
Another purpose-built tool to manage code contributions is Gerrit. Gerrit is heavily backed by git workflow and may not be relevant, but I mention it since it's one of the first such peer-review systems for software.
As described on the Gerrit website, a similar workflow is followed:

Making the Change
Creating the Review
Reviewing the Change
Reworking the Change
Verifying the Change
Submitting the Change
Next Steps

Much of the actual work here is done by the reviewer or contributor, and Gerrit serves to organise their collaboration.
I suspect that neither of these corresponds exactly to what you are asking for, though, however a general purpose tool to which arbitrary contributions can be submitted is a big ask and likely a mirage.
It seems more feasible that this should be achieved by adopting shared conventions in existing code collaboration platforms rather than a dedicated service to manage ingestion.

Finally, you mention:

Usually when maintaining a system...

If you meant "maintaining a configuration state of a system", perhaps an approach with Zuul. Zuul is

... a program that drives continuous integration, delivery, and deployment systems with a focus on project gating and interrelated projects.

While quite heavy, it is a means to actually execute and evaluate the result of a proposed script. This provides more meaningful feedback, but is of course more onerous in terms of setup and maintenance.


How to set minikube to auto start on windows log on?

Should the normal startup functionality of w10 work?

To change which apps run at startup, press and hold (or right-click) the Start  button, select Task Manager, and then select the Startup tab. Choose an app, then select Enable or Disable.
To add or remove an app from the Startup tab, press the Windows Logo Key  + R, type shell:startup, and then select OK. This opens the Startup folder. Select Start . Find the app you're looking to add or remove, and press and hold (or right-click) it. Select More > Open File Location. In the file location folder, press and hold (or right-click) the app and select Copy. Paste the shortcut to your app into the Startup folder to have it run at startup.

Info from:
https://support.microsoft.com/en-us/help/4026268/windows-change-startup-apps-in-windows-10


Jenkins - Scripted Pipeline - Input String

After some more research I found the answer:

  def coreImageTags = input(
    id: 'coreImageTags', message: 'Enter a comma separated list of additional tags for the image (0.0.1,some-tagname,etc):?',
    parameters: [
      [$class: 'StringParameterDefinition', defaultValue: 'None', description: 'List of tags', name: 'coreImageTagsList'],
    ]
  )

  echo ("Image tags: "+coreImageTags)

Important note

It is stated in the documentation but I want to point it out specifically: If you copy/paste my solution here, coreImageTags will contain a string directly. If you do multiple parameters like so:

    parameters: [
      [$class: 'StringParameterDefinition', defaultValue: 'None', description: 'List of tags', name: 'coreImageTagsList'],
      [$class: 'StringParameterDefinition', defaultValue: 'None', description: 'Something else', name: 'somethingElse'],
    ]

You will not get a single values but an array of values:

  echo ("Image tags: "+coreImageTags['coreImageTagsList'])
  echo ("Image tags: "+coreImageTags['somethingElse'])

The following posts will help:

stackoverflow.com read-interactive-input-in-jenkins-pipeline-to-a-variable

CloudBees - Pipeline-How-to-manage-user-inputs

A list of classes you can use in this way and what they do is in the Jenkins Input-Step documentation:
Pipeline - Input Step


How to run "docker/compose" container on RPi3?

According to this reply it should run by issuing:

docker run -ti --rm -v `pwd`/docker-compose.yaml:/docker-compose.yml -v
/var/run/docker.sock:/var/run/docker.sock docker/compose:1.16.1 up


Pros and cons of different deploy strategies

Assuming there are no changes in Master that are not in your release branch and you don't rebuild after you merge the code then you could deploy first and then merge to master.  If either of those are not true, then merging first would be more common.

The process depends on how you handle your branching strategy.  If other things could be merged into master that are not part of the release branch then you need to merge first and rerun the pipeline of tests.  If master represents production and all new work is now in the release branch then yes, you can deploy then push, or do them as a step in the pipeline assuming the deploy is successful then do the merge.


Two identical Azure deployments; one always throws 404 - how to find out what the difference is?

We found out what was wrong. It was not an Azure issue. The problem was that we were deploying a bad ZIP file. The ZIP file was missing web.config, which meant that the web application could not start up.
We were zipping our published web application by having this in the CSPROJ file:
<Target Name="ZipOutputPath" AfterTargets="Publish">
  <ZipDirectory SourceDirectory="$(OutputPath)\publish" DestinationFile="$(MSBuildProjectDirectory)\kmswebapp.zip" Overwrite="true" />
</Target>

This turned out not to work because the compiler does things in a different order than we expected. At the time when it generated the ZIP file, web.config was not generated yet, so web.config never got packed into the ZIP file. Hence Azure could not start the application.
When we deployed from our local machines, it worked because we didn't clean the publish directory before each run, so there would be a web.config left over from the previous run, and this old (but unchanged) web.config would get packed into the ZIP file and deployed to Azure, so Azure would know how to start the application.
We solved it by removing the above from our CSPROJ file and doing (roughly) this in our Jenkinsfile:
powershell "dotnet publish ./src/webapi/WebAPI.csproj"
powershell "if (!(Test-Path('${publishDirectoryPath}/web.config'))){throw 'We need web.config to exist in the publish directory'}"
powershell "Compress-Archive -Path '${publishDirectoryPath}/*' -DestinationPath './src/webapi/kmswebapp.zip' -Force"

This generates a proper ZIP file including web.config, and Azure can now start our application so it can respond properly to requests.


How to start a service in fedora container without using systemd?

Running systemd inside the container should not be required as docker run eventually does the same thing. Make changes in your run scripts to make them independant from systemd.
If you still insist on using systemd and do not care about how docker works, there is a way to let systemd work inside the container. Follow this link:

https://developers.redhat.com/blog/2014/05/05/running-systemd-within-docker-container/


The sign to hire a CIO is that your CEO is not able to handle the relevant tasks on his own, plain and simple. The reasons for that may be manifold - maybe he has too much on his plate, as the company grows, or he does not have a sufficiently technical background, etc.


If you mean removing a "snapshot" of an analysis according to the most recent SonarQube documentation you should be able to this.  Also if you have access to the database SonarQube is using then you could remove the analysis directly from there.  That option may introduce other issues though depending on how SonarQube handles it.


What is a gating continuous integration (CI) system?

A traditional CI system is reactive, it detects regressions, but requires human intervention for repairs.

Traditional CI relies on automated quality checks performed on the already integrated code - after changes are committed/merged into the integration branch (typically triggered by those events). The results of these checks indicate if the integration was successful, i.e. if the branch at that point is a good candidate for any subsequent steps towards delivery/deployment or if regressions occurred. Some regressions are considered catastrophic (for example build failures) and must be repaired ASAP as they're effectively blocking development on the branch. Unfortunately in most cases the repairs require human intervention, which can be a significant bottleneck.

By contrast a gating CI system is proactive, it prevents regressions and, with proper orchestration, can be entirely automated.

Gating CI performs the quality checks on candidate changes prior to them being committed/merged into the integration branch. If the checks are successful the changes are committed/merged into the branch, ideally in an automated manner, performed by the CI system itself, without human intervention. If the checks reveal regressions considered catastrophic/blocking the candidates are rejected (i.e. their commit/merge into the branch is prevented). Hence their gating characteristic - they will prevent "bad" changes from even reaching the branch.

A simple example would be a pull request (PR) based integration with one or more automated validations (for example a build and maybe a smoketest) hooked up in the PR processing which must be successful for the PR to be merged. A fairly common practice for many development teams.

Unfortunately rejecting "bad" changes alone is not sufficient to ensure the integration branch maintains a minimum quality level at all times.

It is always possible to have multiple candidates outstanding at the same time. But having multiple quality checks (on different candidates) active at the same time means that at least some of the checks do not take into account the other candidates in flight, leaving room for regressions due to changes interfering with each-other - see a simple example in How can successfully pre-verified changes cause regressions that should have been caught?.

It is for this reason that those using a PR-based system as mentioned above can probably recall that the branch can still see breakages once in a while.

This can be addressed with proper orchestration of the quality checks. The purpose of the orchestration is to completely eliminate the risk of interference between changes by controlling when each check is started/stopped and what candidate change(s) it covers, based on one or more inputs which can include: other candidates submissions/current states, results of other checks, actual commits/merges, availability of resources necessary for the checks, etc.

A very basic orchestration algorithm for the PR-based CI system mentioned above would be, for example, to only allow the merge if the tip of the branch at merge time is the same as it was when the automated PR validation started (i.e. no other change was committed/merged in the mean time). Effectively serializing PR processing, it could work great for small teams, but probably will not be able to keep up in a large scale project.

The orchestration algorithm (or lack thereof) is ultimately what dictates a gated CI system's scalability, performance ability to maintain the integration branch' quality level. An algorithm 100% efficient in preventing blocking regressions would no longer require human intervention for normal operation, a gated CI system using such algorithm can typically be entirely automated - a pipeline accepting change candidates at one end, filtering out regression-causing changes on a mid tap and outputting guaranteed stable branch tags at the other end.

It's probably worth mentioning that using a traditional or a gating CI only covers the integration stage and brings no fundamental restrictions on the CD (delivery/deployment) portion of a CI/CD pipeline - any traditional CI/CD tool can be plugged in behind a gated CI to perform additional (but non-gating) CI verifications (if needed) and the CD steps. Similarly any release strategy (see Should I use release branch to push my changes?) can be used with either of them.

Note: for distributed version control systems like git the context of the answer is the master/origin/central repository, not the local replicas.


Need a folder, always updated with the latest code from Git

Use a Git post-receive hook to issue a git pull in the directory you want to keep updated. You might want to mount the directory you want to keep updated locally to the Git server, or execute that command remotely with ssh into Git Bash on Windows, or Powershell. Alternatively have the hook call a webhook and have the webhook execute the git pull. The webhook method will be easiest with BitBucket, I don't know about the others.


What is the difference between DevOps and GitOps?

GitOps is a technique of using Git to manage infrastructure provisioning and software deployments.  This technique uses many of the features of Git such as Pull Requests to manage and trigger deployments, and allows "diffing" (looking at the difference between commits) to view the differences between deployments.

DevOps is more of a culture or philosophy.  It is not tied to the specific tool Git.  Instead, DevOps is the joining of Development (Dev) and System Operations (Ops).  Often continuous integration and continuous deployment are big focuses of companies that invest in a DevOps culture.

A company that has a DevOps culture may choose to use a technique like GitOps to manage their infrastructure.  They may choose to use other tools though.

Note: DevOps is not a process. It is a culture. GitOps is a specific System Operations (ops) process tied to a specific tool (Git). A company that uses GitOps may not have a DevOps culture. Conversely, a company that has a DevOps culture may not use GitOps.


Cassandra DB: store binary file record via shell

The only way to do it with cqlsh is wrap binary file into CSV file and run cqlsh --some_args -e 'COPY table_name FROM binary_wrapped.csv', but in order to do so you have to generate a new primary key to use in CSV-file.

IMO: Since cqlsh is a wrapper around python client, your ops should just write simple python script for this operation, it would be much easier in many ways.


Can use a private IP address as the puppet certname?

You can indeed use IP address in the "certname" directive of puppet.conf file.

In addition, you can create a private hosted zone in AWS Route53 that is called for example lab.local and then create an A record for your puppet server such as for example puppetmaster.lab.local and then use this address in the certname directive.

Opening a hosted zone in Route 53 doesn't cost much... around 0.50$ per month for your first 25 zones.

The full Route53 pricing can be found here.


One azure-pipeline.yaml for staging and production

Ok, found it here:
Build.SourceBranch

The branch of the triggering repo the build was queued for. Some examples:

 - Git repo branch: refs/heads/master
 - Git repo pull request: refs/pull/1/merge
 - TFVC repo branch: $/teamproject/main
 - TFVC repo gated check-in: Gated_2016-06-06_05.20.51.4369;username@live.com
 - TFVC repo shelveset build: myshelveset;username@live.com

When your pipeline is triggered by a tag: refs/tags/your-tag-name
When you use this variable in your build number format, the forward slash characters (/) are replaced with underscore characters _).

Note: In TFVC, if you are running a gated check-in build or manually building a shelveset, you cannot use this variable in your build number format.


How to optimize blue/green AWS deployments?

bitbucket deploy: 45 seconds

This seems correct. Not sure of your repo size, but if it is hosted outside of your region, that's not out of the ordinary.

  bitbucket deploy: 45 seconds

Again, seems correct. Not much room for optimization here.

  provisioning a new ec2 instance: 3:40 minutes

What size of instance are you provisioning? Are you provisioning one or more instances? The obvious solution would be to use in place deployment vs blue-green, but I think the extra time is well worth it. If this is for production/staging, I think you are fine. If this is in a testing environment, and you are aiming for quick feed back, you could consider provisioning new instances after a successful blue-green so that, if there is no AMI change, they are ready to go for installation immediately.

  installing application on replacement instance: 40 seconds

Nothing to optimize here

  2 minutes rerouting traffic from old instance to new one *this one is extremely long, if it were done manually it would literally take 10 seconds)

My guess is that step is performing extra steps that you do not perform when you manually reroute traffic. It's possible that CodeDeploy is performing extra checks to make sure that connections are draining, new connections are established before old ones are closed, and other test/checks. This is probably your best area to optimize, but not at the sacrifice of checks and balances.

  17 seconds terminating second instance

Nothing to optimize here.

Summary:
I don't think it is feasible or worthwhile to attempt to optimize your deployment further. The amount of effort it would take and sacrifices you would need to make in your test and checks far out ways the potential minute or two you would save in my opinion. I also don't think it is possible to achieve a <1 minute deployment. That would average ~15-20 seconds a step, which would require a lot of sacrifices in your deployment strategy.

If your goal is for quick iterations for QA, code testing, integration testing, etc., and you absolutely want sub 1 minute deployments, I recommend creating a Cloud Formation stack with your exact deployment except skipping the blue/green deployment model for an in place deployment model. This way, you can quickly see the results of your code. You should then later run your full suite blue/green deployment after your testing in either another testing environment or your staging environment.


For building use Google Cloud build. For deployment you can you use Google Cloud Deploy.
https://cloud.google.com/deployment-manager/docs/
https://cloud.google.com/cloud-build/docs/


The best way to share volume with config data between multiple Docker containers

One could use read-only volumes.

docker run -d \
  -it \
  --name=nginxtest \
  -v nginx-vol:/usr/share/nginx/html:ro \
  nginx:latest

If one would like to use the same volume in multiple containers then one could specify the following when running each container:

-v vol:/usr/share/nginx/html:ro

Note: one could use ro to ensure that the volume is read-only to prevent that multiple containers will write at the same time to the same volume and prevent issues.


Schedule Jenkins Job with Active choice Reactive parameters and Reactive reference parameters

Although I haven't tried using the Parameterized Scheduler in particular, I don't think you will be able to use this plugin to trigger a build with Active Choice parameters - most plugins don't support passing Active Choice parameters to builds.  There is currently an open bug report on this issue.

As for a workaround - I have been able to successfully trigger builds with Active Choice parameters using the HTTP REST API /build endpoint (a bit counterintuitively, NOT /buildWithParameters).  So for instance you could have a second job that runs on a schedule and does nothing but hit the API endpoint to trigger the job and pass parameters, or you could have a plain Unix cronjob that hits the endpoint, or something else along those lines.


There should be a log group available in CloudWatch that you can refer to for information on the error. Also, there is an Execution History tab available on the left hand side that will let you see any errors that are not logged to CloudWatch.

From the sounds of it there may be a problem with the source repo you have selected. It should only take a few seconds to read the repo. Very generally with AWS, many of the problems can be traced to IAM permissions.

You can cancel the execution by selecting 'edit' while it is running.


Ansible Remote provisioning not working in Docker container build

It appears that you are trying to run ansible locally inside the container but instead you are running it locally on the machine you use to run packer on(the provisioning one).

To run ansible in local mode you need to update the ansible part of your packer file to:

    {
        "type": "ansible-local",
        "playbook_file": "./kubeadm.yml"
    }

Also, I would suggest to add connection: local to your ansible playbook, to avoid unnecessary attempts for ssh connections.

- name: install kubeadm
  hosts: localhost
  connection: local
  roles:
    - { role: djx339.k8s-kubeadm-install }


Continuous Integraton using Jenkins Pipeline for a .NET project

Specifically in answer to your 3rd question, if you are willing to look outside of the Jenkins Ecosystem there are alternatives out there that might be of value to you.

For my clients who use the Microsoft Stack and have fewer than four teams, I have been recommending the use of AppVeyor it is highly tuned for the .NET Stack and integrated very naturally with msbuild and Wix.

There are two ways of configuring AppVeyor, either through the web-based user interface or via the appveyor.yml file checked into the root of the git repository.  I would strongly recommend the latter, you are welcome to start from this template:

version: 3.0.{build}

build:
  parallel: true
  project: EVEMon.sln

cache:
  - packages -> **\packages.config

install:
  - nuget restore

The major drawback of AppVeyor and it's cousins TravisCI and CircleCI is they don't play particularly nicely with on-premise source code management solutions, you do really need to be using GitHub or BitBucket.


What are Design Patterns for creating images?

The general design pattern is one container = one running service - see the initial comment at https://docs.docker.com/config/containers/multi-service_container/

Now, after spending about a year developing a multi-service application relying heavily on docker this is what works for me:

Avoid creating files inside the container unless it is your storage engine - in which case use volumes (recommended by Docker and also common sense) - we use Elasticsearch for that.

If I need multiple custom configurations I am trying to either load it from somewhere (DB, Environment) or I mount the configuration as a volume like

-v config.yml:/usr/src/app/config.yml

In general, we have one service = one repository, but sometimes you have to copy all the dependencies in order to just create one script. So we have one repository where these scripts are aggregated and I change the CMD with an environment variable like this:

CMD cd /usr/src/app && python -u $RUN_SCRIPT

It can be probably improved with a combination of ENTRYPOINT and CMD as described here http://goinbigdata.com/docker-run-vs-cmd-vs-entrypoint/

Then we use various docker-compose.yml files for getting the setup we need.

For some operations (like DB setup) we either run one container like

docker run --network ournetwork init-profiling

or I can run a command inside that container

docker run -it --network ournetwork init-profiling bash
# run_command.sh
^D

We spent some time minimizing the footprint of containers, they can quickly get bloated. For example, we use a common base image for our python containers. Also, things like pip freeze do make sense only if you use a separate virtual environment for each repository/container.

We do not use swarm/kubernetes for now, so I cannot add any additional guidelines for that.


Terraform: Sanitize a map that may be a map of strings or map of maps or combo of both

Following the documentation a map is a collection of key/values with a constraint:

"...accepts any element type as long as every element is the same type" source

So the input map you have declared, is already an invalid map for Terraform 0.12 (at least)
Below the example of the error produced:
variable "cidrs" {
  description = ""
  type        = map
  default = [
    ["1.1.1.1"],
    ["2.2.2.2", "3.3.3.3"],
    {
      C = ["4.4.4.4"]
      D = ["5.5.5.5"]
    }]
}

When validating an error is raised:
$> terraform validate

Error: Invalid default value for variable

  on vars.tf line 65, in variable "cidrs":
  65:   default = [
  66:     ["1.1.1.1"],
  67:     ["2.2.2.2", "3.3.3.3"],
  68:     {
  69:       C = ["4.4.4.4"]
  70:       D = ["5.5.5.5"]
  71:     }]

This default value is not compatible with the variable's type constraint: map
of any single type required.

Been that said, if we can generalize A and B as "values which are lists" and Group1 as "values which are maps" and the input is a fixed type ordered we can use a tuple or  an object, this can be solve in a more readable (and maintainable) way. If so, let us know so we can extend this answer.


Applying ansible playbook to a docker container on a remote host

With the help of @Levi (refered to Stackoverflow) I managed to find a way to connect straigt into the docker containers using ansibles docker connection driver and the remote API capability of Docker.

First of all you have to expose the API which is by default not the case. Simply add -H tcp://0.0.0.0:1337 to the ExecStart constant of the systemd script. Then reload the start script and restart the docker service. (From this blogpost)

Then you can list the containers in the inventory file like this:

[containers]
container-name ansible_connection=docker ansible_docker_extra_args="-H tcp://1.2.3.4:1337"

And execute any playbook like your used to.

ansible-playbook -i docker_inventory playbooks/my_playbook.yml

The upside of this way of implementing it this way is that you can mix up "real" hosts/vms that are maintained through SSH and Docker containers. One negative thing I have to mention is that the docker API is much slower than using SSH as a control tunnel.

This setup is of cause horribly insecure and its only purpose is to explain the concept. Before using it in production you should read into the security features and limit access to the daemon as far as possible.


If all the files are there without any extension then yes you can execute the cat command with * (cat ) it will display the content of each and every files underneath this directory, but here in your output it has the zip files (.gz) too, so you need to execute 2 commands rather than single command...

cat syslog syslog.1
zcat *.gz

Hope this will helps.


Vagrant box take long time to access shared /vagrant folder

It seems ther reason for this was that the SMB connections would disconnect.
using

net config server /autodisconnect:-1

in a "run as administrator" command window fixed it.

as explained here:
https://www.vagrantup.com/docs/synced-folders/smb.html#preventing-idle-disconnects


Assign DNS name to Docker Swarm containers with public IPv6

Assuming that "publicly available" is on the public internet it will be difficult.

DNS isn't really that reactive to changes and is challenging to run in a secure fashion on the public internet.

It also depends on what protocol you are running, because of an oddity around SRV records where pre-existing protocols were excluded from supporting it in the RFC.

But in general the public and the private solution I would implement are similar with an additional layer for a public internet solution. (assuming no use of a 3rd party or geo specific answers)

On the internet edge use Unbound DNS server on the edge, and have it relay queries to consul which would handle service registration.

If this is for private usage this would allow for some data-center awareness and fail-over.

If it is just for internal use you can just use consul and either configure your internal DNS servers to delegate subdomains or forward to the default domain.


nginx for dummies? How to enable Jenkins over https

Your question is very broad, but I'll try to answer it.

You really need to read on what TLS/SSL is and how it works before you continue. You can find a bunch of questions in security.stackexchange.com that ask about this and many of them even link to RFCs where you can read the full detailed explanations of how it all works.
To configure nginx to work with TLS/SSL certificate you can use https://www.digitalocean.com/community/tools/nginx to generate the template. Just follow the guide and it should create a relatively secured vhost. If you would like to enhance the security even further - you can follow cipherli.st's guide. They update it relatively often so the proposed settings are usually up-to-date.
What makes a certificate "valid" or "invalid"? It's the Certificate Authority. It is possible to create a private CA certificate and use it to sign a Certificate Signing Request (or CSR). The CSR can then be signed by the CA and returned to whoever needs it (in this case the Nginx server). There are multiple tutorials for this available online and you can use easy-rsa to do the heavy lifting. However, the down side of doing it this way is that you need to go over all machines in the network and install the CA in browsers and all necessary keystores. Otherwise those machine will treat the site's certificates as "untrusted" because it will be coming from an unknown Certificate Authority. In some cases you can also create a Certificate Bundle for nginx.

If you had an ingress available you could have used LetsEncrypt to automatically issue certificates and skip complicating your setup by having CA and issuing certificates on your own.

Note: Image has been borrowed from ssl.com.


Cleanest way to prematurely exit a jenkins pipeline from inside a withEnv

Just the way you can exit any stage prematurely by using return keyword, you can use the return keyword (outside of the stage) to exit the entire pipeline prematurely. You can re-use the currentBuild.result value outside of Stage 4. See below.

node{
    withEnv([...]){
        stage('1'){
        ...
        }

        stage('2'){
        .....
        }

        if( env.BRANCH_NAME != 'master' ) {
            currentBuild.result = 'SUCCESS'
            return
        }

        stage('3'){
        ...
        }
    }

    stage('4'){
        withEnv([...]){
            ...
        }
    }

    if(currentBuild.result == 'SUCCESS') {
        return //this will exit the pipeline
    }

    stage('5'){
        ...
    }

    stage('6'){
    ...
    }
}


JAVA_HOME error in Sonar Scanner for MSBuild

I have added "C:\Program Files\Java\jdk-12.0.2" as value of JAVA_HOME environment Variable and its working. I don't know exactly this is the correct solution OR Not, but when I have checked sonar-scanner.bat file then it's taking %JAVA_HOME%/bin/java.exe location of Java.


How to `terraform destroy` an `aws_instance` without also destroying its `aws_ebs_volume` (separate resource)

This may be what you are doing already (in which case this sounds like a bug), yet it seems just based on the limited information in the question that you may be forgetting a step: Run plan -destroy -target=... -target=... first.  The terraform destroy command will try to operate on changes listed in the .plan file.

The following example assumes a Terraform module named foo, with 3 EBS volumes and attachments named as: dev-sdb, dev-sdf, dev-sdg.  All resources are assumed to have count = local.enabled ? 1 : 0.  So this is the more advanced case you may encounter where each resource would have to be targeted with the array syntax [0].

First create a targeted plan file with terraform plan -destroy -target=....
Do this before trying to run terraform destroy

terraform plan -destroy \
               -target=module.foo.aws_volume_attachment.dev-sdb-attachments[0] \
               -target=module.foo.aws_volume_attachment.dev-sdf-attachments[0] \
               -target=module.foo.aws_volume_attachment.dev-sdg-attachments[0] \
               -target=module.foo.aws_instance.foo[0]  \
               -target=module.foo.aws_iam_instance_profile.foo[0] \
               -out=intermediate.plan

Run terraform destroy, optionally also using -target=... for safety precautions if you wish:

terraform destroy -target=module.foo.aws_volume_attachment.dev-sdb-attachments[0] \
                  -target=module.foo.aws_volume_attachment.dev-sdf-attachments[0] \
                  -target=module.foo.aws_volume_attachment.dev-sdg-attachments[0] \
                  -target=module.foo.aws_instance.foo[0]  \
                  -target=module.foo.aws_iam_instance_profile.foo[0] \
                  intermediate.plan

If you need this to be unattended for automation, add -auto-approve flag to the terraform destroy command.  Otherwise, it assumes an interactive terminal session and will prompt before destroying any resources.


SaltStack: Use Temporary Scripts And Not Cause State Changes To Reported

You might want to consider adding an unless or onlyif requisite in the first directive - especially if script.sh is particularly complicated or expensive. With unless or onlyif, you will use the reults of several BASH commands to determine if you need to push down and execute your script. If your system is out of plumb only then is the state enforced. If everything checks out OK, then the script will never be pushed and executed preventing changed=2 in the final status report. You will want to use a require requisite for the second and third directives to prevent them from running if the script is not pushed by file.managed otherwise you will get to errors instead of two changes.


My approach nowadays, always package your build process into Dockerfile. I.e. here is Dockerfile for one of my open-sourced Go projects:
https://github.com/relizaio/relizaGoClient/blob/master/Dockerfile
Note: if you're using go modules (which seems to be the case), replace RUN go get -d -v ./... with RUN go mod download in the build-stage. BTW, missing go mod instruction might be the reason for your build failure in the current setup.
Then in your CI, simply do docker build instruction, and then copy your artifact from the container if you need it outside.
Key benefit: you can verify build process against same Dockerfile locally and you can port it to any CI tool whatsoever.


what is difference between "sudo shutdown now" versus "aws ec2 stop-instances"

Just to state the obvious for junior folks who might come across this: sudo shutdown now requires access to the operating system (via SSH keys, inbound security groups, etc), while aws ec2 stop-instances is an AWS CLI command which simply requires API authentication (via IAM credentials or STS tokens).
Aside from the obvious difference in invocation, the shutdown is the same.

When you stop or hibernate an instance, we shut it down. You can restart your instance at any time. Before stopping or hibernating an instance, make sure it is in a state from which it can be restarted. Stopping an instance does not preserve data stored in RAM, but hibernating an instance does preserve data stored in RAM. If an instance cannot hibernate successfully, a normal shutdown occurs.

Source: stop-instances documentation
The --hibernate flag defaults to false, so a normal shutdown is triggered immediately by your CLI command.
When --hibernate is true, the suspend-to-disk process differs from a regular shutdown (such as sudo shutdown now), and is further documented here.


How to trim spaces for parameter value in jenkins pipeline?

Use the Groovy trim() method.  For instance, def myTrimmedParam = params.myParam.trim().


Recipe for hiring a winning DevOps empowered team?

I believe DevOps is orthogonal to your question, i.e., it changes nothing compared to a "classical" approach (or to hiring any team at all, not only for software development). You identify what your key needs are (for example, an "architect" who is able to structure large software systems; some "hacker" who is able to fix a Kernel driver if needed; a "tester" who likes to test and such; and finally maybe a "DevOps engineer" who excels in creating good CI/CD tooling).

But all of them need, to stay with your image, to be able to kick the ball. I.e., they all need to work together in the context that the team is working in. They all need at least some basic understanding of what the other team members are doing; if you have a strict CI/CD pipeline, then they all need to be able to develop in that frame; and so on.


On Azure, how do I run a short-lived Docker container on a schedule?

Azure Container Instances
 (ACI) may be a good option as you suggest.  These let you run a container directly on Azure, without having to manage a VM, with per-second billing for the time the container is used.

Although one of the demos on that blog mentions Kubernetes, the idea of ACI is that you can create a container through the Azure CLI with az container create, just like on your local workstation with docker create.

To create the container, you can use Azure CLI (az command, see quick start docs) or Azure Cloud Shell.

You would need to create/run the container on a schedule from somewhere else - Azure Functions might be a good place to run the "container create" command from  a scheduled function.  This supports bash, PowerShell, and other languages - all running on Windows.

If you want to keep using Docker containers without running VMs or learning Kubernetes, this might be a good option.

Alternatively, you could move all your code into Azure Functions, but that's a bigger decision.

Update: Jan 2019 - Azure Logic Apps can be used to run scheduled tasks as well.


The first thing you'll want to do is look for errors.  It sounds like you have a Dockerfile that is creating an image, but when you run the image it appears to not do anything.  There are two immediate things you can do to troubleshoot this:

Do not use the -d flag when running the image.  This flag runs the container in detached mode and will not display container output to the console.
Examine the logs from the container.  Even though the container does not appear to be running, it is most likely crashing on start.  You can verify this by using docker ps -a which will show you all containers (including ones that have stopped due to crashes).

Once you know what the error is you can look at whether this is a problem with your startup script or the Dockerfile.  Based on what you've described so far this sounds like you could be missing files in the right location, or the Entrypoint or CMD is incorrect.


kubernetes quota vs limit. What are they? How are they different? When should I use one or the other?

ResourceQuota is

the total amount of memory and CPU that can be used by all Containers running in a namespace

LimitRangeis

if ... the Container does not specify its own CPU limit, then the Container is assigned the default CPU limit

Example of ResourceQuota:

Imagine you have 2 namespaces in the same cluster, the namespace "backup" and namespace "test". "backup" run critical pods and you want to guarantee that will always have enough resources (CPU and memory) to run it. So you can configure a ResourceQuota in the namespace "test" to use only a part of the total amount of CPU and memory in your cluster.
Is like you say to k8s: "The sum of all memory and CPU of all pods in the namespace "test" can't be higher than what is specified in the ResourceQuota, 20G for memory and 12 vCPU's"

Example of LimitRange:

You can specify in the yaml the minimum and maximum amount of CPU and memory a container can consume but instead of configuring in every yaml file you can set a default value with LimitRange.
Is like you say to k8s: "If there is no resource configured in the yaml file apply this LimitRange.


How to create high availability databases?

There are docker images for running opensource databases in high availability mode such as https://info.crunchydata.com/blog/an-easy-recipe-for-creating-a-postgresql-cluster-with-docker-swarm

You can also find examples of Microsoft SQLServer container for Linux being run with high availability on Kubernetes https://docs.microsoft.com/en-us/sql/linux/tutorial-sql-server-containers-kubernetes

Your question is missing an important concept from the devops perspective. If we run a particular version of a database (MySQL, PostresQL, SQLServer) on either a physical server, or a VMware ESX virtual machine server, or an openstack Xen virtual server, or in Docker in a laptop or in an http://cri-o.io container runtime in Kubernetes it’s “the same software”. If the high availability is based on running two or three instances of the database, and them opening tcp sockets to exchange data, and perform leader failover, then as “it’s the same software” it doesn’t matter whether we are running the database cluster as physical, or on a virtual machine hypervisor, or in a container orchestrator. That means you can learn how to configure the software for HA as containers on your laptop to then know how to configure it for HA on physical servers.

Obviously you might get different performance on different hardware or virtual machines or container runtimes. Typically  it’s disk bandwidth and memory that makes the biggest difference for a database not how the process is being run (within reason). Yet from the perspective of making a high availability setup if the software you are using to only needs normal disk and normal tcp connections you can expect it to work anywhere that lets you access disk and network. It is then a matter of making sure that you provide enough cpu, memory, disk and network of sufficient reliability to meet your NFRs and SLAs.

A lot of companies might setup virtual machine farms or container orchestrators while not thinking about the demands of relational databases running in high availability modes. For example if you configure a HA setup of two Postgres instances but they get placed in two different virtual machine on the same physical server then you have a very poor setup with respect to the likelihood of one machine fault killing both database servers. If you configure all disk to be NAS network drives then that might be fine to run application servers but terrible to run databases under load. Often people are only thinking about scaling up application servers and having many of them crash and restart on different machines as the aim without the same disk being attached at startup. Anyone trying to run a database cluster in such an environment is going to have many sleepless nights. This means it’s important to ensure that wherever you run your databases is sympathetic to the needs of databases to have relative stability. If the environment is optimised and operating as though everything hosted is a stateless application server workloads that is not fussy about being moved around and restarted at random you might regret setting up a database cluster there.


Sonarqube stage in jenkins pipeline fails with 403 error

Problem:
The execution user/group had permissions to run  “Execute Analysis”
Solution:
Update the default template with permissions and push to projects (Bulk Apply Template)


TravisCI build continues to fail even after changing alpine version

You cannot rely on FROM node:10.15-alpine alone to fix this. I had to go with the fix from the facebook team. So if anyone else did not get a fix from modifying your Dockerfile to node:10.15-alpine, go to your package.json file and change react-scripts to the following facebook fix:

"react-scripts": "3.0.0-next.b0cbf2ca"

Then go ahead and remove your node_modules/ folder via rm -rf node_modules and then run npm install, check your jest-util file, it should now have an output like this:

└─┬ react-scripts@3.0.0-next.b0cbf2ca
  ├─┬ babel-jest@24.5.0
  │ └─┬ @jest/transform@24.5.0
  │   ├─┬ jest-haste-map@24.5.0
  │   │ └── jest-util@24.5.0  deduped
  │   └── jest-util@24.5.0
  └─┬ jest@24.5.0
    └─┬ jest-cli@24.5.0
      ├─┬ @jest/core@24.5.0
      │ ├─┬ @jest/reporters@24.5.0
      │ │ └── jest-util@24.5.0  deduped
      │ ├─┬ jest-runner@24.5.0
      │ │ └── jest-util@24.5.0  deduped
      │ ├─┬ jest-runtime@24.5.0
      │ │ └── jest-util@24.5.0  deduped
      │ ├── jest-util@24.5.0  deduped
      │ └─┬ jest-watcher@24.5.0
      │   └── jest-util@24.5.0  deduped
      ├─┬ jest-config@24.5.0
      │ ├─┬ jest-environment-jsdom@24.5.0
      │ │ └── jest-util@24.5.0  deduped
      │ ├─┬ jest-environment-node@24.5.0
      │ │ └── jest-util@24.5.0  deduped
      │ ├─┬ jest-jasmine2@24.5.0
      │ │ ├─┬ jest-each@24.5.0
      │ │ │ └── jest-util@24.5.0  deduped
      │ │ └── jest-util@24.5.0  deduped
      │ └── jest-util@24.5.0  deduped
      └── jest-util@24.5.0  deduped

You should be good to run TravisCI build now. The issue is the react-scripts version 2.1.8 is conflicting with new version of Node. Suffered through this all day long, grateful to the facebook team for getting back to me.

For reference: https://github.com/facebook/create-react-app/issues/6475


Terraform | Upgrade to 0.12 error Reference to "count" in non-counted context

It looks like your underlying goal here is to retrieve the full set of attributes for each of your subnets that are marked as Tier = public. Here's a different way to do that using features from Terraform 0.12.6:

data "aws_subnet_ids" "public" {
  vpc_id = data.aws_vpc.vpc.id

  tags = {
    Tier = "public"
  }
}

data "aws_subnet" "public" {
  for_each = data.aws_subnet_ids.public.ids

  id = each.value
}

With the above, you should find that data.aws_subnet.public is a map from subnet id to the attributes of that particular subnet.

The reason for the error in your case is that you used count.index but didn't set the count argument, and so there is no count index to return.

The count-based equivalent of the above, which is more similar to your original example, would be this:

data "aws_subnet_ids" "public" {
  vpc_id = data.aws_vpc.vpc.id

  tags = {
    Tier = "public"
  }
}

data "aws_subnet" "public" {
  count = length(data.aws_subnet_ids.public.ids)

  id = sort(data.aws_subnet_ids.public.ids)[count.index]
}

The result of this is very similar to the for_each-based example I gave above, but with one significant difference: in this case data.aws_subnet.public will be a list of subnet objects, ordered by the lexical ordering of their subnet ids. That will usually make the result harder to use elsewhere in the configuration, so I'd suggest using the for_each approach unless there's a specific reason why you need a list.

(If you use a map and then later find that you need a list in a specific context, you can always use values(data.aws_subnet.public) to take the values from the map, in the same lexical order.)


All you need is a backup of the postgres database and the projects (to be safe but they should be rebuildable by pulling them again from your VCS in most cases).

Exporting an ovf of you full vm is really an overkill IMHO.
Copying the volumes (pgdata and projects) is by far the simplest solution (the one I actually use). You simply have to put the folders back in place, and docker-compose up -d the correct version and you are set. But to make sure your backup is consistent, you need to stop AWX before copying the dirs, especially for postgres.
Dumping the postgres database and copying the projects will let you do a backup while AWX is running. For your recovery, you will have to first pop-up a posgres container and reinject the dump before starting all the other containers.

Keep in mind that AWX is the dev version of Tower. As such, there is no real "recommendation" for backups. So the above explanation simply reflect my own experience with AWX. The commercial Tower version embarks the backup in it's deployment playbook. I have never played with it.

If you did not experienced with an AWX upgrade yet, you should be aware that they are sometimes working but can be a real pain (export from old version / import to new version / re-set all the user passwords and jobs credentials). So a backup to quickly rollback when testing the upgrade is vital in this case.

Hope this will help you make a wise choice ;)


Are using TestContainers safe for integration tests?

I think the question about Docker security in general was already well answered in other places, for example here: https://stackoverflow.com/questions/40844197/what-is-the-docker-security-risk-of-var-run-docker-sock

In your particular instance, I see the following points:

The tests should run on some testing server, possibly even in a VM that is destroyed/reset after each test run, so possible accidents/incidents should not affect other systems
For an attacker to use this setup, they would probably (depending on your tests) need to be able to modify your test code and if they are able to do so, you will already have much bigger problems than privileged Docker access


Sharing Gems between Chef and System Ruby

To take your points one by one:

  but in the interests of good system hygiene I would like to avoid as much as possible duplication

Can you ensure your system ruby will evolve at the same pace as the Chef's ruby ? Main point of chef coming as a bundle with its own ruby comes back from Chef 10 which was installed via a simple gem command and was usually a problem switching between ruby needed for applications and ruby needed by chef, the same goes for gem dependencies. I would advise against mixing system and chef ruby.

  the Gems I have in the system Ruby come from the known-good local Yum repo and are installed via RPM over the LAN whereas chef_gem goes off to the public Internet

While I understand and agree with the worrying about recent pipy events, there's two things here:

Your system ruby install gems from yum repository because your system ruby's gem command has been tweaked for that, that's specific to your distribution. -
Chef's ruby use usual ruby method going to rubygems.org, as any ruby installed by rbenv or any other way than yum install would do.

If you really want to use a local repo you'll have to set up a gem server (alternatively, some artifact repositories can handle that also) and set-up the sources for gem using chef's gem sources in /opt/chef/embedded/bin.

If you're using a chef client above 13.0 your can use the rubygems_url in client.rb to tweak that:

rubygems_url

  The location to source rubygems. It can be set to a string or array of
  strings for URIs to set as rubygems sources. This allows individuals
  to setup an internal mirror of rubygems for “airgapped” environments.
  Default value: https://www.rubygems.org.
  Changed in Chef Client 13.0.

And finally:

  I have tried setting ENV['GEM_PATH'] in my recipe both before and
  inside a ruby_block but this appears to have no effect, I get a
  failure on require

Chef omnibus package hardcode some values specially to avoid interfering with the system ruby, I think the GEM_PATH environment variable is ignored when searching for gem you require within the recipe code. I don't think there's a workaround for this, and again you're heading to a compatibility problem on mid term as you'll have a drift between chef's ruby and system ruby.


The same as any Linux load, consider no more processes in wait state than the number of CPU.
The load on 1, 5 and 15 mins given by uptime should ideally be 1 less than number of cores.

Containers are roughly isolated processes, leaving a core for orchestration avoid congestion.

That doesn't mean only 7 containers on a 8 cores machine, it's a matter of simultaneous load, so the best way is to measure it and adapt consequently.


How comes a GitLab CI/CD job cant' find its artifacts?

As Casey Vega commented: provide the complete path.


Looping an Array with Missing Keys in Ansible Tasks

Q: "How could I get it to fall back to Ansible's default for keys that don't exist?"
A: Omit such parameters, e.g.
    create_home: "{{ item.create_home|default(omit) }}"


How to produce a map from the cartesian product of two lists to use with for_each on a resource with Terraform

In 2 steps, you can first create your cartesian product in a variable:
locals {
  datadog_gke_clusters_by_pov = flatten([
    for cluster in var.datadog_gke_clusters : [
      for pov in var.gke_slo_ : {
        cluster    = cluster
        pov        = pov
      }
    ]
  ])
}

And then use this new variable in your resource block:
resource "datadog_synthetics_test" "gke-monitoring" {
  for_each = {
    for valuue in local.datadog_gke_clusters_by_pov : "${valuue.cluster.name}-${valuue.pov.zone_name}" => valuue
  }
...
# here you can use each.value.cluster.* and each.value.pov.*
}

Have a good day!


Jenkins declarative pipeline: use branch if it exist

There's a resolveScm step implemented by the Pipeline: Multibranch plugin:
checkout resolveScm(source: [$class: 'GitSCMSource',
                             credentialsId: '<credentialsId>',
                             id: '_',
                             remote: 'https://your.git.remote/repository.git',
                             traits: [[$class: 'jenkins.plugins.git.traits.BranchDiscoveryTrait']]],
                    targets: [BRANCH_NAME, 'master'])

Above procedure will look whether $BRANCH_NAME exists, check it out if yes and check out master otherwise.


How do -dev apk packages differ on Alpine for the purposes of creating docker images?

dev packages are typically intended for development and build purposes. They usually contain header files and other resources for providing programmer APIs. For example, python-dev contains C headers for C/C++ applications that integrate python functionalities (which of course isn't necessarily python itself).

The "non-dev" packages provide the actual application or library: for example, python provides the python interpreter binary, the libpython3.so shared library, and compiled python modules.

For inspecting the contents of specific packages, you could use Alpine's excellent package explorer: locate the package of interest (for example, python3-dev), and click the "Contents of package " link at the bottom, to see the complete list of files installed by the package.


Can't hit redis using DNS name in multi-container pod on Kubernetes

You're using two different host names:
$ rdcli -h dbbackupd.default.cluster.local
$ host dbbackupd.default.svc.cluster.local

The one used with rdcli is missing .svc.


DevOps vs SRE vs Production Support Engineers

The best answer to DevOps vs SRE is here What's the Difference Between DevOps and SRE ? From the above series, you will understand Class SRE implements DevOps and both are working on a similar ground and it's pretty align goal for both of them.

As a production support, you will be doing the task which you have mentioned in the question, apart, you need deep visibility, alert and notifications, latency, metrics, tracking, security, etc. lots of things to take care. And, it basically providing the service to the end users who is utilizing your application or system. Also, I see production support team have written the dockerfile and bash scripts to automate many tasks. So, whenever user faced any issue that first comes to Production Support team and it might went to other IT teams if it's not easily resolve. But, if you look at SRE, their sole purpose is to keep system in reliable state every time, maintain resiliency, etc. In some organization, the DevOps does this part too.

From my perspective, all the three terms are same as their goal align and just differ in functionality. These functionality differs from organization to organizations. But, DevOps are intended to bring culture and shift in paradigm so that everyone will work collaboratively. Hope this helps.


AWS Production Configuration ASP.NET Core & Docker

If you are going full on AWS, I would suggest to have a look at AWS SSM parameters. When EC2 or ECS container are starting, you have a script that is going to fetch the parameters. Then you can do whatever you need to do: replace variables in config files with the parameters fetched previously for example.
You could also use files stored in S3. You would be able to copy it and place it in the right application folder when your EC2 or ECS container are starting.


How to avoid a regular user can only test in production?

A Sandbox could be part of the solution

To bring material to the subject you can check the questions with tag sandbox on MetaSE, there are sanboxes for:

Q/A formatting.
Comments formatting.
Chat.

There's a feature request about "Can we get a Stack Exchange sandbox?" which has no status (neither declined, pending, review nor in-progress meta-tag).

So I assume the answer (to the specific sample in the question) is that there's no way to test out of real SE sites for now and my recommendation to SE would be to accept this feature request.

When is a sandbox appropriate?

I don't think it's always needed to give users a sandbox, and it can be overkill sometimes. Defects should be spotted on Q/A environments, and when they're not, well, either you can know there's a glitch by a fine monitoring of your logs, or you will just never know until someone tells you.

Sanboxes come at an additional cost

Even if you deploy a sandbox for your users to try and confirm a bug, nothing tells you they will inform you of a bug/problem and you had to go through adding a public environment with dummy or anonymised data with no idea of it's ROI.

So It boils down to: "You can create a sandbox for your users but it adds complexity to your infrastructure for a hardly measured return"

Recommendation

My personal opinion would be to put the efforts in log analysis to spot unwanted behaviors instead.


Cassandra persistence with Docker - is an example of a comprehensive, up-to-date and facts-based guideline for a deployment architecture available?

Since you posted your original question, Datastax has come out with dockerized images of datastax enterprise, along with a comprehensive guide. I'd start with this reference and maybe check out this older "Best Practices" doc, although some details might have changed.


How to mark a check as warning or error if a check fails multiple times within a certain time range using Sensu?

You want to use the occurrences filter, which is included by default in Sensu.

First, apply the filter to your handlers:

{
  "handlers": {
    "email": {
      "...": "...",
      "filters": ["occurrences"]
    }
  }
}

Then, in your check, you can use the occurrences attribute to only trigger the handler once the check fails or warns a certain number of times in a row:

{
  "checks": {
    "check-http": {
      "...": "...",
      "occurrences": 2
    }
  }
}


Yes, you can rename an RDS instance... at your own peril.

  When you rename a DB instance, the [DNS] endpoint [hostname] for the DB instance changes, because the URL includes the name you assigned to the DB instance.

  http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_RenameInstance.html

Your application code requires configuration changes to be coordinated, for this reason.

See the link above for the other implications.  This is a supported and generally safe operation, but only in your maintenance window, because there will necessarily be a brief disruption as the DNS is changed and your app code is cycled in with the new database hostname.


Can I bypass host key checking in Ansible?

This is typically done by setting the following value in ansible.cfg:

[defaults]
host_key_checking = False

If you don't want to modify ansible.cfg you can set an environment variable like so:

export ANSIBLE_HOST_KEY_CHECKING=False

Source: http://docs.ansible.com/ansible/intro_getting_started.html#host-key-checking


Strategies to work around 30 sec timeout

A variation of method 2 is infinitely scalable:

write the data to s3.
from s3 trigger lambda to break s3 file into sqs queue events of good size
from sqs queue trigger lambdas to insert
sqs can be fifo queue if it matters.


Try to check is registry really stopped using docker ps -a if container runned stop container then start.

Part related to the --restart always option, restart always will proceed only if container works at least 10 seconds and docker itself can monitor this service.

See more info at this link restart policy detail


I have an custom AMI and want to run certain config commands at initial boot, depending on the autoscale instance that starts it. How do I do this?

If I am understanding you correctly, you have one AMI used in several autoscaling groups and want to run a specific “bootstrap” sequence on it depending on the autoscaling group it belongs to.

This is not difficult to accomplish because the launch configuration you created to map the autoscaling group to the AMI can accomodate user data which can be a shell script.

In the shell script provided as user data, it is customary to retrieve meta data so that the instances can correctly identify itself in logs and arrange other similar last minute details.  This meta-data could also contain a reference to the auto-scaling group, which you could take advantage of to reduce the little overhead due to autoscaling-group management.


fatal: ambiguous argument 'master': unknown revision or path not in the working tree

The error looks like you're using a non-existent reference (branch/tag). Try specifying the correct branch and tag using the branch/tag options. You can also use the commit hash directly using ref option.

Also ensure that your git URL is correct (your code does not contain the xxx.git part).


How to mitigate "[DEPRECATION NOTICE] registry v2 schema1 support will be removed in an upcoming release" in quay.io?

You don't need to do anything; this warning is a mistake. See https://github.com/moby/moby/pull/39736


Are configuration management tools appropriate to use as deployment tools?

In such context the typical advice should be immediately applicable: use the right tool for the job.

But then you also cannot ignore nowadays the almost virulent tendency of software tools to extend functionality into more or less related fields and actually become toolsets for various reasons: cool feature(s) to have, expand customer base, amass more revenue, etc.

For example many file management tools include image viewing features and many image processing tools include file management features. You can move files around and you can view images with either of the tools, often equally well.

Because of this it's quite possible to have entire portions of the software development process covered/overlapped by multiple tool(sets) even if their main feature/capability differs.

So it really boils down to the exact functionality you want to achieve in your particular process and how well one tool or the other does the job in your context. Subjectivity/preferences/convenience included.

Making this question primarily opinion-based ;)


How should I store my environment variables?

I've had two runs at doing environment variables in a scalable way and neither has ended up perfect because,  as I've discovered, is a very tricky thing to get right. I'll give a summary of both of my experiences below:

Common Factors

Environment variables are stored in a separate repository from the original source code (they are submoduled together but are still based on separate repos)
There is a separate "build" process for the artifact and it's variables.
There is not a separate release process for the environment variables. If you want to change environment variables you need to go through the same change review boards and usual

Using Consul KV Pairs

The environment variables are loaded from an artifact repository (never the original git repo) and loaded into a namespaced KV pair tree, for example

/env/dev1/my/application/v1.1.1

Where the preceding dev1 is the name of the environment, the my/application is the application namespace, and the v1.1.1 is the version of the environment variables to use.

To developers all these things are invisible. At runtime the platform checks that the environment exists in the current consul cluster (if it doesn't there's a problem and it's errors out), it then checks the subtree for the applications namespace (this way there can be no cross contamination where one app references another apps vars) then the version number of the configuration is taken from label connected to the deployable artifact. Updating this label is the key thing here because it means if we lost both production data centers we could stand up the environment again by simple reading the meta data from our deployable artifacts and loading all the environment variables into the KV store.

Problems With This Approach
Developers always, and I mean every single time, found a way to slip configuration changes into the environment that had significant impacts to how the application would run. Because it always ended up being easier to get configuration changes approved than code changes.

Storing a "Deployment" Artifact with variables embedded

This tightly couples the exact version of the artifact to the version of configuration. If you changed configuration then you had to rebuild this deployment artifact.

The deployment artifact itself was essentially a yaml file which contained the URL to the releasable binary and all the configuration attached to it.

The platform contains componentry to read variables and then out them into the process tree of the application when it starts up.

This has so far been a lot more successful because there is an artifact we can trace the history of and which we can hold up to any review board and say "this is the only artifact we care about, we don't need t look at any other changes, only changes to this thing" (ie. Version of application to deploy,  environment variables included etc.

This makes it just a little bit harder for developers to try and build logic into their application that will change its behavior based on variables so they can slip in changes without going through appropriate testing cycles.

Bonus Points

Consider application secrets. Our solution to this has so far been to provide a public RSA key which development teams use to encrypt an extended Java key store (almost every language has a library somewhere that can read Java keys stores) this is then treated like a third type of artifact and is pulled onto the server, decrypted with our platform private key, and provided to the application at run time.

Admittedly secrets management is its own can of worms. But it's probably worth considering.


You have 3 methods to get secrets to an app inside a docker container. The first 2 involve docker configuration. The last one is to have your apps directly fetch secrets from a secret store.

1 - Environment variables

According to "The 12 Factor App" guide, secrets are merely config, and they should always be set in the environment. You could set your secrets as environment variables during the docker run, and your app accesses them from there.

2 - Mounted volumes

You could have your secrets all within a particular configuration/secrets file, then mount that to your instance as a mounted volume.

3 - Fetch from secret store

As @030 mentioned, you can use Hashicorp Vault (or "Amazon Secrets Manager", or any service like that).
Your app, or a sidecar app can fetch the secrets it needs directly, without having to deal with any configuration on the Docker container. This method would allow you to use Dynamically created secrets (a very appealing feature of such systems) and without having to worry about the secrets being view-able from the file system or from inspecting the env variables of the docker container.

Personal Opinion

I believe env variables is the way to go. It's easier to manage, and you can still pull from a secret store like Hashicorp Vault, if you have you CI build system pull the secrets during the build and set them when you deploy. You get the best of both worlds, and the added benefit of your developers not needing to write application code to fetch secrets. Devs should be focused on their code functionality, and not dealing with admin tasks like fetching passwords.

Your application's code should be focused on it's own app functionality itself, and not dealing with backend tasks like fetching passwords. Just like the 12 Factor App states.

Edit: changed last sentence to remove implication of Developer vs SysAdmin silo-ing. The tasks themselves should be separate from a code perspective, but DevOps is about the same persons keeping both in mind and not be limited.

Personal Opinion (Update)

Per @Dirk's excellent comment (Passing secrets to a Docker container), there is a very strong argument to prioritize a secret store over ENV vars, due to not wanting to leak them.


Jenkins multi branch pipeline configuration for bitbucket

I recommend using the BitBucket Branch Source Plugin when using BitBucket. Documentation can be found here


How to get the AWS Access key of an already created Access ID

Access Key is that AKIA... one. What you need is a Secret Key - that can't be recovered after the initial setup.

Since you have never used this particular access key (it says "Last used: N/A") it's safe to Make inactive and then Delete and then create a new one through Create access key button above the form.

Hope that helps :)


What courses would be useful for a DevOps Engineer?

Linux Academy has several webinars and tutorials (including sandbox VMs for hands-on experience) in several technologies favored by DevOps shops including AWS, Docker, Ansible, Jenkins, Puppet and many more.


What is the need of plugins in Jenkins (Sonarqube)

Any CI/CD plugin can ultimately be scripted.  However, using plugins gives you better centralized control.

Consider the following scenario: Your SonarQube server location has to move to a new location.

If you are using a script like the example you provided above, you would need to manually update every script.  This process is error prone as you may miss a script or worse, type the wrong address.  Instead, if you are using a plugin you can update the server location once and be done.


Is it possible to recover accidentally deleted HashiCorp's Vault secrets?

If your secret backend is a separate service, which it should be, then with regular backups you should be able to stop vault, replace the backend with a backup and be good to go.

I've done this for a file backend but don't know what would happen for other backends so ymmv.


How to implement Continuous Deployment in Angular, PHP API and Ansible projects?

Let's assume you have the following scenario: You have many customers, but most use the exact same app, except for some configuration changes. You would like each of your customer to get the latest of the app ASAP.

If this is your scenario then try make everything that needs to be differentiated between the apps a configuration change. You build one (or one set of) docker containers that can handle any of your customer based on the configuration it was given. Then on deployment time you simply supply the proper configurations for this container for each of your customer.

The benefit of this approach is that you don't really incur massive overheads when a new customer comes. You generate their configuration, create the new servers, add some entries into an inventory file, and the next time you deploy, it gets there as well. The more well your CD pipeline is set up the more you can automate any of this process.

So how would the flow generally look like:

You have your application codebase, that can cater for all of your customers
You build the docker container for your application (one for the frontend, one for the backend, possibly a few more supporting ones)
You have a deployment script that uses a configuration file to determine where to deploy and what.
You run this deployment script. It will go over all of your customers, and simply deploy the latest version of your app with the appropriate config.

If you have a new customer, the only place you have to change is the deployment configuration file. If you are using ansible most likely you have to change your inventory and your vars file. Let's say you have get a new customer, who would like the app to run under Spanish locale, and has access to the beta features. This customer's configuration would maybe look like this:

config.js:

SETTINGS = { locale: 'es', beta: true };

config.php:

define('CUSTOMER_LOCALE','es');
define('CUSTOMER_BETA','true');

You then make sure that during deployment your container gets these files mounted on startup.


Building Jenkins pipeline with GitLab info

Gitlab Subgroups are essentially only additional path identifiers on the Gitlab server.

For example if you have:

Gitlab server on localhost
Group: test
Subgroup subgroup1
Repository myawesomerepo

Then the path to your repository will be:

http://localhost/test/subgroup1/myawesomerepo

And cloning the repo will be

git clone git@localhost/test/subgroup1/myawesomerepo.git

So when you connect any Jenkins pipelines to the repo you will need to connect them to that full path.

Any changes being done or API's added to the Git repository itself can easily be monitored automatically to do some action via Jenkins.

However, since a Gitlab Subgroup is not a Git feature rather a Gitlab feature, you need to try and write you own logic to query on the full URL and to see if there were any new Subgroups/ API endpoints added.


Cancel a Spot Request vs Terminate Spot Instance

You can definitely cancel your spot instance requests via the Management Console and via the CLI. There appears to be a command to terminate the instance also.

See the following AWS docs as a reference:

AWS Docs - Cancel Spot Instance

As for terminating just the spot instance, I'm not sure but I think the, still active, spot request would spin up the instance once the spot price matches the request again.

** Edit **

Worth noting that if you terminate the spot instance within an instance hour then you pay for the full hour, even if you only used a few minutes of it.


Coincidentally I had to do something very similar yesterday. This is the script I used:
#!/bin/bash -e

VAULT_NAME=...   # put your Backup Vault name here

for ARN in $(aws backup list-recovery-points-by-backup-vault --backup-vault-name "${VAULT_NAME}" --query 'RecoveryPoints[].RecoveryPointArn' --output text); do
  echo "deleting ${ARN} ..."
  aws backup delete-recovery-point --backup-vault-name "${VAULT_NAME}" --recovery-point-arn "${ARN}"
done

You can also add --by-created-before to delete the old backups only.
Hope that helps :)


How to remotely update AMI id in Jenkins EC2 plugin?

The PR you are linking to contains a link to a python script that does exactly this, since you can run that script from anywhere you can access your Jenkins server.


Reset time vs alerting window size in alerting strategies for SLOs

Consider a service using the alerting rule from the book:

job:slo_errors_per_request:ratio_rate1h{job="myjob"} > 36 * 0.001

The burn rate is the rate at which we are exhausting our error budget. A burn rate of 1 means that we'll have exactly exhausted our error budget at the end of our reporting period. A burn rate of 2 means that we'll have exhausted our budget half way through our reporting period.

For a 99.9% SLO, an error rate of 0.1% constitutes a burn rate of 1. An error rate of 1% means a burn rate of 10, and a 100% error rate means a burn rate of 1000

If we want to alert when we've exhausted 5% of our 30 day error budget in 1 hour, then that means we are burning error budget at a rate of 36, which translates to an error rate of 3.6% for a 99.9% SLO.

Imagine a service with a constant rate of requests, that starts serving 100% errors at t=0 minutes

Here our SLI counts the total number of "events".

SLIs written in terms of some kind of discrete events, where an event can either be good or bad, have many nice properties: They range from 0-100%, where 0 is "nothing is working" and 100% is "nothing is broken"; I find this quite intuitive. By making all of your SLIs behave the same, you simplify tooling, alerting rules, and understanding.

With a request-based system, events are usually individual requests. So 0 errors, 1 request in minute at t=-2. I used 1 request per minute for simplicity. The result is the same if you have 1 request per minute or 1 million; it's the ratio of errors to total requests that matters.

t=-2, errors=0, total=1
t=-1, errors=0, total=1
t=0, errors=1, total=1
t=1, errors=1, total=1
...

If our monitoring system evaluates its rules every minute, we get:

at t=-1, our 1h average error rate is 0
at t=0, the 1h error rate is 1/60 = 1.666%
at t=1, the 1h error rate is 2/60 = 3.333%
at t=2, the 1h error rate is 3/60 = 5%

Our alert fires 2 minutes after the event started.

Let's assume our incident continues until t=60.

at t=60, our 1h error rate is 100%: a burn rate of 1000, for a 99.9% SLO
at t=61, the 1h error rate drops to 59/60 = 98.333%
at t=62, the 1h error rate is 58/60 = 96.666%
at t=118, the 1h error rate is 2/60 = 3.333%

At this point, our 1h error rate is below the alerting threshold, and so the alert is no longer in the firing state, 58 minutes after the end of the event.

at t=120, our 1h error rate finally drops back to 0%, as no errors have been observed in the last hour.


How to properly setup a multi-environment release pipeline in Azure YAML pipelines?

Assuming that deploy to DEV and QA steps look a lot like your deploy to production-staging environments, put the code that does that work into a yaml template and take advantage of parameters to allow you to pass in differing environment settings for DEV, QA, and Staging-Production.

Then in one pipeline create a stage for each environment.  The stage should have a job which references your template.  You should use conditions on each stage to ensure your code goes to the environment when you want it.  Here's a pseudocode example:

stages:
- stage: MyDevStage #No condition so this stage always runs
  jobs:
    - template: '../MyTemplate.Yaml
      parameters:
        ServerName: MyDevServer
- stage: MyProdStage
  condition: and(
    succeeded() # if my dev deploy doesn't go well don't deploy to PRD
    , eq(variables['Build.SourceBranch'], 'refs/heads/master') # confirm this is a commit to master
    )
  jobs:
    - template: '../MyTemplate.yaml'
      parameters:
        ServerName: MyProdServer

At the top of your pipeline yaml file you'll need to specify a trigger which may look something like this:

trigger:
  batch: true
  branches:
    include:
    - master


Can I use docker compose to run application container and its dependencies on google cloud or digital ocean, the same as locally?

If you don't want/need platform-specific integrations and transforming your services to a cloud I guess GCP Compute Engine is for you -- it's like a usual virtual machine that you can start, SSH to it and have all privileges to install and run software the same way you did it on your dev machine.


Azure DevOps Artifact.zip folder structure for .NET Core 3.0 application

I'm looking this fact:

  I have an ASP.NET Core 3.0 application for which I have a full CI/CD setup in Azure DevOps. The Application is hosted on a Ubuntu 18.04 server machine.

Therefore I can safely assume that you are developing ASP.NET Core 3.0 app to be hosted in Ubuntu. Any .NET Core 3.0 (or later) application means that you should rely on the dotnet build instead of using VSBuild.

Also you had stated that you will host the app on Ubuntu 18.x, then you should also run the build on Azure DevOps agent that runs on Ubuntu. This means you should only use dotnet build in DotNetCoreCLI@2 task, because VSBuild task only runs on Windows based agent, not Ubuntu and it is intended to compile .NET Framework and other platform other than .NET Core.

Please consult the official doc of DotNetCoreCLI@2 task at https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/build/dotnet-core-cli?view=azure-devops

After you use dotnet build, use dotnet publish to publish your artifact. It is much easier and it's also the best way to publish .NET Core apps.

I have sample usage of dotnet build and dotnet publish of this on my repo:
https://github.com/eriawan/dotnetcore-cicd-samples/blob/master/yaml-samples/sample_pipelines_check.yml


How to avoid wasting time/resources in a CI/CD workflow?

We break ours down into stages, and each stage pulls, rather than pushes.

Our build takes 20 minutes. So if I commit once a minute, I will still only get three builds an hour, but each build will contain 20 commits.

The tests then pull on the latest successful builds and run them, and the same for the deployment scripts.

The main point is that we do NOT build every commit individually. We do NOT test every build individually.

I probably got all this from Continuous Delivery by Jez Humble and David Farley.


How to skip reviewer for project administrator pull request?

As a workaround, you can add "bypass policies" for project admins.

The documentation: Security groups - Git repository (object-level)


API gateway and service discovery on Amazon AWS

It turns out there is no existing solution for such a configuration.

I have to develop something on my own.

A nice source for help might be a series of articles on the Amazon developer blog and REST API documentation for the Amazon API gateway service.


What are the reasons Docker should not be used for databases?

When people talk about running a database in Docker, they do not mean to store the data in a container; they are talking about having a docker image with the DB software, and mounting the data as a volume (a bind volume, not a container volume).

Volumes are an essential part in Docker, and are not something that is flakey or just tacked on. Docker is not just made for stateless (micro)services.

Wish as I might, I cannot find a technical reason not to run a database in a Docker, so unfortunately I'll pick the other side of the argument and hence maybe not give you the answer you are looking for.

(I'm using Oracle as an example because I'm familiar with it, both bare metal and dockerized, and because it's quite a notorious beast for being just a bit non-trivial to operate if you go past default settings.)

Packaging up the DB software itself in a container gives you the usual benefits - having the same version everywhere, avoiding dependency/shared library issues, being able to spin up the exact same DB on developer laptops or wherever you need it.
It is a snap getting it to run anywhere; updating is trivial, and so on. All the Docker benefits apply. There is an Oracle image on Dockerhub which allows you to spin up a working DB in a minute or three (and for the others as well, of course).
People did do performance tests and found no I/O differences between volumes and bare metal (https://www.percona.com/blog/2016/02/11/measuring-docker-io-overhead/, https://stackoverflow.com/questions/21889053/what-is-the-runtime-performance-cost-of-a-docker-container).
Under the hood, it's not like Docker somehow intercepts all I/O, anyway. It just gets creative with standard Linux tools (bind mounts in this case, mangling of the internal kernel tables that make the Docker-fu possible at all).
Obviously that does not mean that you can run two instances of the DB and just have them work on the same files, but nobody is implying that. Docker does not give you automatic simultaneous and magically race-free access to volumes, and did never pretend to do so. The rest of the benefits still apply. If your DB itself does not detect conflicts like this, you better supply a CMD script to the image which refuses spinning up a second container when the volume is already in use.
You have to be a little more careful spinning up/shutting down the container (just as you would not simply switch off a bare metal DB server), but that should be quite manageable.

Now, depending on circumstances, there may be soft reasons not to do it:

Oracle (the company), for example, will certainly not support you if you run their RDBMS in a Docker container. But maybe you are using dockerized Oracle RDBMS images only for your developers and the testing environment, where you would not need their support in any case, reserving it for a bare metal production server. (But don't forget to pay your licenses...).
If the ops guys are unfamiliar with Docker, it might just be a bit easier to accidently kill everything, destroy your data files etc..
If you have big dedicated metal DB machines already, with large amounts of very fast dedicated SAN storage, and running nothing else anyways, then there would just be no point in using Docker to containerize those as you will never just spin another server up when there are 100s of GB or even TB of data. After all, for production, a RDBMS like Oracle is very, very advanced in all the replication, data integrety, no-downtime failover, etc. aspects. Note that this argument just says "you do not need to containerize your RDBMS". It does not say "you should not do it" - maybe you want to do it because you wish to roll out database software upgrades through containers or for whatever other reason you could imagine.

So there you go. By all means do dockerize your DB, at the very least for your developers (who will be eternally thankful) and your testing environments. On the production, it will come down to taste, and there at least, I would also prefer the solution that sits best with the specialized DBA/Ops - if they have decades of experience working bare metal DB servers, then by all means trust them to continue so. But if you are a startup who has all IT in the cloud anyways, then a Docker container would just be one further piece of onion in the whole picture.