Artifact is an abstraction. It could be anything that's produced by somebody else. In Bulgaria for example we're making artifacts for the winter time ( it's called 'compot') . Basically it's an artifact, it was produced by my grandmother and could be used by me :)

Same goes into the IT world, but with one small difference.

The Java artifacts are a little bit more special, since Java is a static language and before the run phase you would have those compiled artifacts - *.jar files. After that you could reference them during the 'run' phase. This is the reason why they are always pointed as the most straightforward artifact example, the java compiler will produce them.

One major distinguishment between my grandmother's artifacts and Java's ones is that it's a good practice to have a version for each artifact, that way you could easily switch them and see what works and what does not.

JavaScript and Python are dynamic languages, they do not have any compiled artifacts like Java, you could just refer to the '.py' file.

As a summary an artifact could be anything:
from tar.gz to a jar file, but it must always be versioned :) Of course the versioning might come during a later stage, like a docker image tag, but the principle still remains.


Does local cache solve the problem with locked files when deploying to Azure App Service?

The answer is no, if you are using the recommended setup, which is:

App Service should have Staging and Production slots
Staging slot has Local Cache disabled
Production slot has Local Cache enabled
The following settings are configured as "Slot setting" aka "Sticky setting" - WEBSITE_LOCAL_CACHE_OPTION, WEBSITE_LOCAL_CACHE_SIZEINMB
The code is deployed to Staging slot
Staging slot is swapped with Production slot

For deployments to Staging, I created the following steps:

Shut down Staging slot by introducing App_Offline.htm (POST to SCM URI or can be part of artifact being deployed)
Wait for slot to stop before cleaning (probe the URL of Staging slot)
Cleanup wwwroot (post to SCM URI) - this is to clean unused files. Deployments are not incremental, so if it's important to have clean old files, you can introduce this step.
Deploy to Staging slot using az webapp deployment source config-zip
Remove App_Offline.htm (POST to SCM URI)

For swapping slots:

Check Staging slot is up and ready (probe the URL of Staging slot) - this one is useful when triggering deployment environments in Atlassian Bamboo when deployed to Staging. You need to wait for Staging to be up and loaded before swapping, otherwise issues may occur
Swap the slots using az webapp deployment slot swap

If you deploy directly to Production slot, and restart it right after (because of Local Cache enabled), then yes, there should be no file locks when using this method. But this creates the downtime during deployment, in contrast with deployments to Staging slot and swapping it with Production.


What are the advantages of using Jenkins over Hudson?

Number of commits.

Jenkins is a fork of Hudson (as commented here) but has a more active development, Hudson's last commit is quite old. And look at Hudson's open bugs


The point of DevOps, is that development shouldn't oppose operations, instead they should support each other.

Traditionally, due to waterfall deployments and large scale updates, development would cause operations a variety of problems when deploying due to inadequate testing, changing server environments, the list goes on and on. Essentially, the updates were too large for the operations team to be able to effectively deploy them without some problems arising in the process. These problems might be why you believe that development opposes operations.

On the other hand, DevOps works to reduce update size, decrease rigid environments, and generally improve the handoff of the application between development and operations by increasing the amount of times the handoff occurs each year. With the increased number of deployments comes less headaches for operations, because they have either automated a large amount of work required to update the products, or they better anticipate and prepare for the updates.

Tldr: DevOps aims to nullify the theory that development opposes operations by creating a mindset where operations and development work together to frequently deploy products in a timely and easily reproducible way.


Container Image Promotion across environments in ECR/ECS

Although difficult to give a canonical answer, I think I understand your frustration. My opinion is  that you are focussing on promoting the wrong thing - the image.
I would start with the concept of "Dev-Prod parity" (or in your case, UAT/Prod parity). This concept says:

Keep development, staging, and production as similar as possible

The simplest way to do this is to use the same image in the two environments. The difference between them comes from the configuration and backing services. These are set at run time, not at build time - ie they are changed when you start the container not when you build it.
So, your process should look like:

Build the image, tag it with uat
UAT environment discovers new image, new image gets deployed
Run UAT against new deploy, they pass
tag that same image with latest
Prod discovers new image, gets deployed.

This process is extremely lightweight and you will have certainty in what is deployed.
TL;DR instead of promoting images and having to rebuild them, just shift the tag used in prod.


How to install curl in conjunction with --no-install-recommends in Debian correctly?

you need just to install ca-certificates so update your apt-get command to :
RUN set -ex; \
    apt-get -y install --no-install-recommends \
        ca-certificates \
        curl=7.64.0-4+deb10u1 \
        unzip=6.0-23+deb10u1 \
        coreutils=8.30-3 \
        lsb-release build-essential ssh-client apt-transport-https \
        python gnupg


I'm guessing that your PowerShell script is hanging because of the Invoke-Command command. My best guess is PowerShell is either prompting you for something, you are passing in an argument incorrectly, or you are executing the script in your script block incorrectly. I'm not sure exactly how Script-Block works when referencing paths to PowerShell scripts, but it could be because C:\hg\Update.ps1 does not exist on the remote server. It could also be because of the commands in your "C:\hg\Update.ps1" script. Some debugging tips that I recommend:

Remove your Invoke-Command line and see if the program executes.
Replace the ScriptBlock command with a simpler command, like Get-Culture, command and see if it still hangs.
Replace your C:\hg\Update.ps1 with another script that performs a simple command.
If all of those options still hang, try using the simple script method, but with different args.

And of course as with all PowerShell remoting issues, you should check that your firewall and network settings on the remote server allow connections from Jenkins over ports 5985 and 5986.


How to add shell script to jenkins freestyle job

You can specify the script in the "execute shell" step of the build. Where it gets executed will depend on what labels the job needs, and which nodes those labels are associated with.

It sounds like you want to execute this script on an agent. As long as the agent is available, you can request it via a label in the job configuration. It's a bit difficult to determine this without having access to your setup, but if e.g. you have a set of agents on Debian boxes, you could attach the label debian to them, and specify that the job should run on debian nodes in the job configuration.

If on the other hand it should be run on master (where Jenkins itself is running), you can specify the label master.

You thus need to put your script into the "Execute Shell" box that Jenkins provides you, perhaps declaring what shell and options you want to use:

/bin/bash -xe
du -sh /bbhome/shared/data/repositories/* |sort -h |tail -20 |
while IFS= read -r line;do
        DIR=`echo $line | awk '{print$2}'`
        Rep=`cat $DIR/repository-config |grep 'project\|repo' |  tr '\n' ' '`
        Size=`echo $line | awk '{print $1}' `
        echo $Size $Rep
done

  I need also to add an ssh command to the environment. This should be executed using passwordless authentication.

For this to happen, you need ssh keys to be exchanged with the host onto which you want to ssh into.


How to get the latest stable version of Firefox?

I pulled this from the Firefox Chef cookbook (in particular this .rb file). This an example of the request they use to get the latest version based on OS and language.

https://download.mozilla.org/?product=firefox-latest#&os=win#&lang=#en-US

It adheres to the following format:

https://download.mozilla.org/?product=firefox-{VERSION_NUMBER}#&os={OS_ABBREVIATION}#&lang=#{LANGUAGE_ABBREVIATION}

I searched around for official api documentation, but couldn't find anything. If I do, I will post a link to it in this answer.


Automating creation of new VM and pushing code to server

Instead of using custom scripts which it seems that you are using, I would consider using open-source tools, similar to what Tensibai has suggested in his comments.

I would further split your two Tasks a bit more differently, such that Step 1 would be to just "Set up the VM", and Step 2 would be to "Provision the VM with required programs and configuration".

My 2-cents would be to use tools such as Packer and Ansible.

You would use (for example), the VirtualBox Packer builder, to stand up your VM, which would handle points 1-4 of your Step 1, and then use the Ansible-local provisioner, or something to this effect, to run a playbook, which would handle the rest of Step 1, and all of Step 2.

The above suggestion would make your life a lot easier, and would be able to abstract different OS's as well as you'd just write a different Packer template and Ansible task. With installing different Python versions / Tools, you'll just need to update the Ansible role / have several different roles with the specific tools.

There are of course, other tools such as Vagrant, Puppet, and Chef, which can handle what you'd like to achieve, so it's up to you find your preferred tool.

Mini breakdown of Ansible modules you can use:

Open up port: win_firewal_rule_module
Installing bitvise and python: win_chocolatey
Setting up pip and virtual envs: pip
Transferring public keys / requirements.txt etc: win_copy

Etc. The full list of Ansible Windows Modules can be found here.


create resource name based on terraform.workspace

Variables are static, literal inputs to your configuration that come from either the command line or from tfvars files. It is not possible to use interpolations here, since the variable values are processed before interpolation begins in order to make them available to other expressions.

If the goal is to define the S3 bucket name in one place and re-use it several times in the same configuration, the Local Values feature can do this if you are using Terraform v0.10.4 or newer. You can declare the named value in one of your .tf files:

locals {
  s3_bucket_name = "tf-${terraform.workspace}"
}

You can then interpolate this value in various places in your configuration using the name local.s3_bucket_name, like this:

resource "aws_s3_bucket" "s3" {
  bucket = "${local.s3_bucket_name}"
  ...
}

Local values are conceptually similar to module outputs but they are exposed within the same module they are defined in, rather than passing a value up to the parent module.

If the goal is to allow this generated default name to be overriden optionally by a variable, then that can be achieved with some conditional logic:

variable "override_s3_bucket_name" {
  default = ""
}

locals {
  s3_bucket_name = "${var.override_s3_bucket_name != "" ? var.override_s3_bucket_name : "tf-${terraform.workspace}"}"
}

resource "aws_s3_bucket" "s3" {
  bucket = "${local.s3_bucket_name}"
  ...
}

The above will generate a default name based on the selected workspace if override_s3_bucket_name is not set, but if that variable is set then its value will be used instead.


What processes enable software vendor/supplier management in a DevOps way of working?

Since you reference the handling of vulnerabilities specifically in your question, I'll speak to this regarding how we handle this within our organization and group of companies.

We have an Information Security department who are responsible to monitoring and detecting vulnerabilities. You could think of this team as being responsible to technical information security management. When vulnerabilities are detected using regular scan with tools such as Nessus or published US-CERT by the information security community, this team passes these vulnerabilities to each Information Security Officer within the group.

The Information Security Officer, who is responsible and accountable for risk management, is tasked at a high level with:

Establishing Information Security Policy, Standards, Procedures and Guidelines
Promoting Information Security Awareness
Completing Information Security Assessments and Audits

Having received the information regarding vulnerabilities, the Information Security Officer evaluates the vulnerability, determines the criticality of the vulnerability and it's impact on the business applications and environment. This information is then given to the appropriate Development Manager.

Development Managers are responsible and accountable for supplier and vendor management. Business Applications are allocated by business domain to various Development Managers. The Development Managers work with either a vendor and/or their own Scrum team to resolve the vulnerabilities withing their domain. Should specialist advice be required, this is provided by the Information Security Officer.

Regarding your possible options.

Scrum teams might well lack the organizational clout to negotiate and compel vendors to address vulnerabilities quickly. It's also unlikely that the scrum teams themselves would have been in a position to ensure that all the correct legal clauses have been put into the vendor and supplier contracts to ensure that there is an obligation to handle vulnerabilities within a given SLA.
Maintaining a central function ought to work to a degree, however as you point out they would lack the full context. Three tiers of management, covering three different aspects will address this short coming.
A 3rd Party automated service will be able to alert you to vulnerabilities, but it will not necessarily be able to correlate those results to a specific business application, and as a consequence a given vendor or supplier. You will still need individual people to manage this aspect.


How to determine whether the Jenkins build was started by a user or the scheduler?

I'm not sure if there is a simple way to check if the "scheduler" kicked off the job, but you can check if a user executed the job.
In your Jenkins job, check for the presence of the BUILD_USER_ID environment variable. If it is present, retrieve the value and send your email to the specific user(s) (scenario 1). If it is not present, a non-user has executed the job, and you can email the users listed on the last successful check-in (scenario 2).
If you are looking for methods outside of the Jenkins job, you can also use the Jenkins API (http://jenkinsurl/job/[JobName]/[BuildNumber]/api/json?pretty=true) and parse the BUILD_USER_ID from the output.


Autoscaling containers using local request counters

The problem can easily be solved using the following components:

On the instances serving your webapp continue to monitor the number of incoming requests – and anything else you see fit.
Publish the number of incoming requests to a monitoring system. If it is not yet implemented, this step will improve your monitoring capacities, and will help you to monitor the load on each hosts as well as host balancing.
From the incoming requests, deduce an estimated required number of webaspp servers needed to serve that work load, as well as the difference between the actual number and the estimated needed number. In the case you describe, it seems that the estimated number is just a scalar function of the total number of incoming requests on a recent period of time. On other systems or after some times, more subtle strategies can be implemented. Monitoring these quantities and the difference could ease the traceability of the auto-scaling strategy, and will monitor its responsivity.
Last implement the auto-scaling itself, at this point, this is really just reading some number from your monitoring system and writing it to your scaling system.


Why it is recommended to run only one process in a container?

Lets forget the high-level architectural and philosophical arguments for a moment. While there may be some edge cases where multiple functions in a single container may make sense, there are very practical reasons why you may want to consider following "one function per container" as a rule of thumb:

Scaling containers horizontally is much easier if the container is isolated to a single function. Need another apache container? Spin one up somewhere else. However if my apache container also has my DB, cron and other pieces shoehorned in, this complicates things.
Having a single function per container allows the container to be easily re-used for other projects or purposes.
It also makes it more portable and predictable for devs to pull down a component from production to troubleshoot locally rather than an entire application environment.
Patching/upgrades (both the OS and the application) can be done in a more isolated and controlled manner. Juggling multiple bits-and-bobs in your container not only makes for larger images, but also ties these components together. Why have to shut down application X and Y just to upgrade Z?

Above also holds true for code deployments and rollbacks.

Splitting functions out to multiple containers allows more flexibility from a security and isolation perspective. You may want (or require) services to be isolated on the network level -- either physically or within overlay networks -- to maintain a strong security posture or comply with things like PCI.
Other more minor factors such as dealing with stdout/stderr and sending logs to the container log, keeping containers as ephemeral as possible etc.

Note that I'm saying function, not process. That language is outdated. The official docker documentation has moved away from saying "one process" to instead recommending "one concern" per container.


Does Kubernetes internal networking keep traffic to co-located Pods where possible?

This looks to be answered on offcail docs here

NodeLocal DNSCache improves Cluster DNS performance by running a dns caching agent on cluster nodes as a DaemonSet.
With this architecture, Pods will reach out to the dns caching agent running on the same node, thereby avoiding iptables DNAT rules and connection tracking.
Having a local cache will help improve the latency.

Image Reference : https://kubernetes.io/docs/tasks/administer-cluster/nodelocaldns/#architecture-diagram


Octopus Deployment - only deploy some specific folders

Yes.. we can control the packages using nuspec file which will be added to your solution and you will be specifying the list of folders/files you dont wanna deploy. And you will use that nuspec to create the nuget package for deployments


Is it possible to load balance TLS/SSL based logstash communication to elasticsearch?

You can do this a couple of ways, but I have never seen anyone do this with a CNAME.

One option, if you have control over your IP addressing is to use BGP to fail-over (or load balance) an IP to your cloud hosting providers.

A second option is to invest in a DNS-based geographic load balancing solution, such as the GTM from F5, the GSLB solution from A10, or the GSLB solution from Kemp (F5 is the most commonly used in the industry). With these solutions you will set your name server (NS1 and NS2) for your domain to be the GSLB solution of your choice. You will set a very low TTL for your DNS records and configure health monitors to your ELK stack from the GSLB solution. These can be as complex as logging into ELK and scraping for data or as simple as making sure you can establish a TCP connection and/or ping your server. When this health monitor fails, the GSLB solution will failover traffic to the alternate data center automagically. You will be able to do this as an HA pair with one GSLB solution at once DC as NS1 and the other at your secondary site as NS2. The GSLB will then forward lookup to your previous nameservers which will publish IP addresses for both sites. These will be rewritten to eliminate the IP that the GSLB does not want to direct traffic to at the time (again, this can be used for active-active sites - you can even use geolocation to send users in Europe to a Eurpoean cloud provider  and US based users to a US based provider, or do a simple active/standby)

A final way to do this might be with local load balancing solution, such as F5's LTM, A10's SLB, or Kemp's solution. You may also be able to use nginx for this for a cheap solution. In this scenario, you might have this solution onsite in an office with the URL to the load balancer served up by your local DNS server to a local IP (a Virtual IP) on your load balancer. You would then configure the load balance to direct users to the desired cloud provider by configuring a health monitor (of whatever complexity). You would then need to configure this load balancer to bridge your SSL connection. You would add your certificate and private key to the load balancing solution. Users would establish an encrypted connection to the load balancer. This encryption would be stripped off by the load balancer and a new encrypted connection established to the cloud provider by the load balancer, and the data forwarded along over the new SSL tunnel.


Best practices on how to erase everything in destination deployment folder and restart the app after deployment using Appveyor

So it actually only took one line of code:

1) First take the SSH public key from the settings page inside the appveyor project and make sure you put it into the authorized_keys inside user_that_you_are_going_to_login_with_from_appveyor/.ssh/authorized_keys

2) To clear the folder before deploying to it on a Ubuntu server

before_deploy:
  ssh user@host -t "rm -r -v /pathtofolder/*;exit; bash --login"

3) Now about the restart of the app I wanted to use after_deploy however there is a problem with that. After deploy fires straight after deploy and doesn't wait untill upload is finished. I wanted to use it to run

systemctl restart myapp

However the problem seemed to solved itself because systemctl restarted my app at the moment when it was substituted with a new one or maybe at the moment when I call it the first time after the files have been reloaded. So far it seems I have a new app running after I run the script.


Tomcat deploy war with Ant - error writing to server

It could be disk space, or hot deploy. For hot deploy, you can put the following parameters:

...
<target name="tomcat-stop">
    <exec executable="${server.home}/bin/catalina.bat">
        <arg value="stop"/>
    </exec>
</target>

<target name="tomcat-start">
    <exec executable="${server.home}/bin/startup.bat">
        <arg value="start"/>
    </exec>
</target>
...
<target name="all" depends="tomcat-stop,clean,init,compile,junit-slow,make_war,deploy,tomcat-start"></target>

The answer is at this link: https://stackoverflow.com/a/32482795/8442153


This job is stuck because you don't have any active runners online

Runners are machines (typically containers) where your jobs run. Gitlab offers shared runners, and you can also bring in your own runners (machines that you register against Gitlab to run your jobs).
The main difference between Gitlab shared runners and the ones you bring is that the shared runners are, well, shared, so sometimes you have to wait for a bit for a job to start. You can also bring more powerful machines so that your jobs run faster.
You can find some docs on runners here.
In your case it seems that the runners have a problem, looking at the exclamation sign next to the name, which should be a green ball. Maybe hovering over the sign will give you some clues into what's happening.


Best way to create a Ci/CD pipeline to reduce bugs and facilitate refactor

It might not be obvious now, but usually as projects advance full regression costs (resources/time) grow much faster than static analysis ones.

You'll also find that static analysis alone isn't a sufficiently solid bug gate, it must be complemented with regression testing as well.

So I'd place the full regression after the static analysis in the CI/CD pipeline - I wouldn't want to waste full regression costs on code which doesn't even pass static analysis.

Down the road you'll probably find that running full regression and/or full static analysis in earlier pipeline stages can easily become bottlenecks:

if you block other commits while waiting for the run results you'd slow down the commit rate
if you allow other commits while a run is in progress they'll change the repository context potentially invalidating the earlier run results (for example you can't immediately fix a runtime bug uncovered by the run if the subsequent commit that happened in the meantime causes a build time breakage).

Addressing these bottlenecks usually means splitting such runs across 2 major  functional stages (each one can actually consist of multiple sub-stages and/or steps):

the (pre-)integration stage(s), which execute subsets of these runs in the earlier pipeline stages:

shorter - to maintaining a sufficiently high commit rate and avoiding bottlenecks
the target quality level is below deployment, but high enough to keep development flowing
fixing regressions detected in these stages has a high priority.
it's possible to use such runs to gate commits/merges in order to prevent rather than detect and fix regressions, in which case these would actually be pre-integration stages

the post-integration stages, executing the longer/full regression and/or static analysis runs:

the goal here would simply be increasing the quality to deployment levels (i.e. blocking the actual deployment if not met)
regressions detected at this stage aren't blocking for development, fixing them is a lower priority


How to structure a IaC for production, and development?

Ok so I understand that you are in AWS and Docker already and Terraform is something you want to bring into picture for IaaC. and Yes, docker for production. Now, let's start with environment:
Assume you have three environments, Dev, Test and Prod. So you create three folders, Dev, Test and Prod which contain infra code for three environments. Now, I don't understand what you mean by infrastructure provided by docker but docker will contain your web servers and web application. Number of containers will depend on how large or small your application is and are you in micro services architecture. So what Terraform does for you is creating almost everything: Right from network structure(VPC, subnets, NACLs, gateways etc), autoscaling groups(I prefer this as you may want to use docker orchestration), security groups, IAM roles, policies to RDS and ECS etc.

So you can tag them in your code as per their environment and you create each of these components using Terraform modules.

So now you have Terraform code which creates infrastructure for you. Store your state file in S3 because you want consistency in your infrastructure state. Keep your database state file separate from other state files. You may want to have separate state files for your network, security groups, RDS, IAM, Cloudwatch etc.

Now comes the deployment part: Since you are in Docker, why don't you use AWS ECS? It will be easy deployment (Almost zero downtime) and orchestration will be managed by ECS itself. So now even deployment you can do using Terraform. Maintain a separate state file for this too.

You can create separate docker images for dev, test and production during your CI. If you do not have any docker tagging strategy, you may use $BUILD_NUMBER:$ENV for now. One of the purpose of using docker is that you don't need to care about where you want to run the container so identical docker images for Dev, Test and Production is almost always ok with some configuration changes.

You do not create anything locally. Use CI/CD tools such as combo of Jenkins and Git which triggers everything.


Push Jenkins Build status to Github and Trigger Jenkins build from Github Webhook

Triggering Jenkins using GitHub webhook should be a straight forward process. This article would help you.


Jenkins Email notification for multiple builds

The other option you could try is to add email notification as part of your build script. I use ANT build script and call the email notification target from my jenkins job instead of using the email plugin


When your micro service in production needs a secret, where is the secret stored and retrieved from?

Short answer

Ideally, you should store secrets as environment variables, and retrieve them from a secrets management system like Hashicorp's Vault or AWS Parameter store.

Long answer

I saw your questions out of turn, and kinda touched on this in your other question:

  Again, there are many perfectly valid options for handling secrets: Chef vault, 1password, environment variables injected by your build instances, home-grown solution using encrypted files stored on object storage, etc....

It really depends on how you run and deploy your apps. I worked at a company that used Amazon Beanstalk for production, and they shipped all their secrets and configs as a 'secrets.conf' file that was created by the Jenkins build server, and shipped with the app when it deployed. The problem there was that it was unruly to manage the same secrets across multiple jobs in Jenkins. So, I came in and moved them towards using Chef to create the file instead, and Chef pulled the file from Hashicorp Vault.

At my current company, they are heavy users of GitLab with it's built-in CI/CD build system. Because the the apps are all Docker instances running in Kubernetes, all secrets are actually set as environment variables set during the deployment. The build server gets those from 'secret variables' that is set from the GitLab UI.

These fit the business need at the time, and that's what really matters at the end of the day. However, you if want to be proper, you should store secrets as environment variables because you specifically mention micro services. It is highly recommend that you follow the 12 Factor App approach to configs: https://12factor.net/config

  The twelve-factor app stores config in environment variables (often shortened to env vars or env). Env vars are easy to change between deploys without changing any code; unlike config files, there is little chance of them being checked into the code repo accidentally; and unlike custom config files, or other config mechanisms such as Java System Properties, they are a language- and OS-agnostic standard.

So you'll definitely want to leverage environment variables. How you set those environment variables could be your Configuration Management system (Ex: Chef), or your build server when it does the deploy, or your apps could be coded to pull directly from your secret storage and set the variables internally.

For actual place where the secrets live (where they are retrieved from), you have a few options. But the 2 you mentioned in your comment would be really great ones to use: Hashicorp's Vault and AWS Parameter store. Others would be Chef Vault, home-grown solution using encrypted files stored on object storage, 1password, etc.... Ideally, you'd want something that can be programmatically access from your build pipeline or apps.


Equivalent of GitLab's "before_script" for Jenkins?

You can use the Jenkins provided step dir() which changes the directory to one relative to the Jenkins env.WORKSPACE (/var/lib/jenkins/workspace/JOB_NAME by default).

stage('Build') {
    steps {
        dir('path/to/dir') {
            // do stuff
        }

        dir('path/to/other/dir') {
            // do other stuff
        }
    }
}


How to list security groups for the specific instance via CLI?

You can use aws ec2 describe-instances instead, you can specify the id --instance-ids <value> and it will output the security group.


Jenkins pipeline pass upstream build number to downstream job using Pipeline script

Remove the quotes around env.BUILD_NUMBER.

Instead of:

value: 'env.BUILD_NUMBER'

Try:

value: env.BUILD_NUMBER


If you want to make it private either you need to setup a vpn so that you can access jenkins gui using http://private-subnet-ip:jenkins_port or you can move jenkins behind a load balancer and use load-balancer-dns to access jenkins gui. If it is in public subnet then use NACL/Security Groups to restrict access only to trusted sources. For more security, do not assign public ip to jenkins machine, assign dns hostname(even to load balancer you should do that) and implement ssl over http so that it can be accessed over https only.


Is OpenStack the same type of software as Azure, AWS, etc.?

Generally yes - all of them are large ecosystems of related software products working together to implement "clouds".

There won't be an exact 1:1 mapping for each and every one of their individual products, features and APIs, so don't expect that. But there is a quite significant overlap in capabilities and functionality in general. Just like with the public cloud providers: they're similar to each-other, but not exactly 1:1 compatible.

You (and your customers) will have to learn the specifics of using the desired cloud products in the ecosystem, just like with the other cloud infra providers.

But you'll also need to learn how to install, maintain and operate the infrastructure in general as well as the individual products you're gonna offer, which is something users of the public cloud providers don't need to worry about.


Cross Account EC2 Access from within Amazon Workspace

Use a VPC peering between the two accounts so that your instance from account A will be able to communicate with account B.
This will work if you have different subnet CIDR blocks in the two accounts. If you have the same (i.e. in both VPC, your subnets are in 192.168.0.0/16), then change the one that has the least impact and proceed with a VPC peering.
Finally, you will have to configure your security group to allow the bastion to communicate with the other instance.
Here are more details for AWS VPC peering: VPC peering


According to this blog post, Docker Desktop on Windows stores its containers inside another container, that container is stored in C:\ProgramData\DockerDesktop. If you docker inspect a container you have setup, it will give you a Linux path, which you can then get to once you go inside the host container using docker run -it --privileged --pid=host debian nsenter -t 1 -m -u -i sh
This assumes you are using the default (Hyper-V) hypervisor for Docker Desktop. Where you use VMware as the hypervisor instead, different rules may apply.


What is the DevOps Best Practices for Hybrid environments?

There may be a significant speed difference between provisioning an environment and bringing up that environment. Especially when it comes to what you mentioned: IaaS and hybrid environments - those are VMs and bare metal servers - it may take many seconds/minutes for some of them to come up.

So there may still be value in patching those environments, provided, of course, that the respective patches can be done without downtime. That depends on the patches - upgrading the kernel on a linux server, for example, would require a reboot to take effect. Upgrading Python won't.

The intended usage for particular IaaS/hybrid environments is also an important aspect in this discussion. The downtime could be completely insignificant for a distributed app designed to tolerate restarts of individual servers in the pool it's running on but probably a big deal for a centralized server app serving thousands of users.


The answer is Key Based SSH

Reasons

You can SSH multiple host machines using the same key or Different keys.
If you reboot your machine while playbook running it again SSH using your provided key so your ansible-playbook runs without occurring any errors.
Also, you get authentication using the appropriate user.

use can use inventory.ini file

example

[hostMachine]
ubuntu@xx.xx.xx.xx  ansible_ssh_user=ubuntu ansible_ssh_private_key_file=/path of your pem file

[ABCDMachine]
ABCD@xx.xx.xx.xx ansible_ssh_user=ABCD ansible_ssh_private_key_file=/path of your pem file

Command for run ansible-playbook

  ansible-playbook -i inventory.ini main.yml


What are your companies policies for S3 data durability?

The durability design objective is not especially relevant above some number of 9's, because it is simultaneously both extremely unlikely and non-zero.

The 99.999999999% durability is easily calculated by reading between the lines as the product of the failure probabilities of 6 copies each with an AFR between 1% and 2%.  Older versions of the documentation suggested 3 AZs × 2 devices per AZ was relevant to the durability calculation.  Assuming these "devices" were RAID arrays, these AFRs are well within reason.

This is raw durability.  It does not take into account natural disasters or acts of war or terrorism or other sabotage (any of which would need to wipe out multiple availability zones within a region) or serious bugs in the S3 system code... all of which are also extremely unlikely but similarly difficult to quantify.

And we need to factor in bugs and human errors in your code and systems, compromise of your credentials, and malicious insiders in your organization.

The bottom line is that the risk of loss is extremely low, difficult to quantify, but still non-zero.

I have many millions of objects (dozens of terabytes) in S3 and have never lost one.  A majority of them have been stored with copies of both their md5 and sha256 checksums, they are all cataloged in an external database, and extensive audits have never revealed a corrupt or missing object.

But if I lose one critical object in the future, all of that still becomes irrelevant.

Can you live with the loss of an object?  If so, no problem.  It's really unlikely that you will lose one.

But if the data is critical, there is a compelling argument that it should still be backed up, and not exist in only "one place."  An object in an S3 bucket is physically in multiple places but logically that is still "one place" because whether the object exists without corruption ultimately resolves to a single true or false answer from your perspective.

Just as using most classes of RAID arrays reduce the likelihood of data loss but do not remove the need for backup, I would argue that S3's durability -- as high as it is -- does not elminate the requirement that critical data be backed up elsewhere.

Cross-region replication is one option, and apparently one important motivation for the introduction of the 1-Zone-Infrequent-Access storage class -- similarly durable except for the case where the availability zone is lost -- was as a very low cost target for cross-region replication.  It seems to meet that objective -- it's more vulnerable to the loss of a single AZ (and you don't get to pick which AZ) but that's only relevant when data loss has occurred in another bucket, in a totally different region.  A replicated bucket could also migrate its copies of objects to Glacier, further reducing long-term storage costs. (Note that very small objects may encounter storage cost increases since 1ZIA has a minimum billable object size of 128K, and with the Glacier storage class, you will be paying for 40k¹ of metadata storage in addition to the object size in bytes).

Depending on the two regions involved, cross-region data transfer charges can be as low as $0.02/GB (or even $0.01/GB for the special case of us-east-1/us-east-2) which hardly seems burdensome if you need that additional level of assurance -- and note that since CRR also supports object versioning and can be configured not to replicate object deletion, it provides some protections against the risks that you pose to your own data -- risks that arguably exceed the systemic risks in S3 itself.

Other options are cross-cloud-provider replication, copying objects to GCS, for example, which would be more expensive due to data transfer costs but then can also fit into your "beyond worst case" disaster recovery plan for when AWS (or your entire AWS account?) is completely inaccessible.  The cost of cross-cloud can be partially mediated by using aggressive compression where applicable, and services like Direct Connect, which reduces the cost of exporting data from AWS, particularly if the presence of the interconnection is justified by other purposes.

Or, consider the extreme opposite: a managed local backup strategy that makes physical, tangible backups on DVD or Blu-Ray is still better than no backup at all.  I take a fair amount of grief for suggesting this "low tech" approach, but as long as the lifespan of the media is factored in and aggressive checksumming and integrity measures are in place, this is a viable protection against the extremely unlikely event where data is lost from S3.  The only problem with this solution is that it appears to cost less than it really does -- it can be somewhat labor-intensive, requiring frequent touching of disks even when everything else is automated -- and of course if you compare the costs of this "simple" solution to the relative set-and-forget of cross-region replication, CRR becomes a pretty obvious choice.

¹ 40k is an oversimplification, based on two values,  8k + 32k, mentioned in the Amazon S3 Developer Guide. "For each archived object, Glacier adds 32 KB of storage for index and related metadata" is straightforward enough, but the explanation of the remaining 8k is a bit less intuitive: "For each object archived to Glacier, Amazon S3 uses 8 KB of storage for the name of the object and other metadata. Amazon S3 stores this metadata so that you can get a real-time list of your archived objects by using the Amazon S3 API... You are charged standard Amazon S3 rates for this additional storage."  When S3 migrates an object to the Glacier storage class, you are still able to access informarion about  the object in bucket listings and access all the object's metadata in real time, just as any other S3 object, even though the object payload itself is stored in Glacier and not accessible in real time. This metadata is stored at the S3 STANDARD storage class rate, rather than the Glacier rate.  It is unclear whether this is a fixed 8k, or actually a maximum, depending in part on the actual size of the object metadata, which includes but is not limited to the optional x-amz-meta-* user-defined metadata for each object.  As such, the importance of this 8k as a cost consideration is not entirely clear.  The 32k is noteworthy whenever it is significant in comparison to the total size of a small object.


Reading two Repos in Azure Build pipeline

I also use the classic editor, and I have implemented this for our builds. My use case was that I have a set of common set of build/release scripts stored in their own repository, and I want the ability to bring them into the application builds that I am executing.

I created a custom Task Group with an Inline PowerShell script to do this:

# Reference: https://blog.rsuter.com/script-to-clone-all-git-repositories-from-your-vsts-collection/
Write-Host "Cloning MyRepo from Azure DevOps..."

$sourcesDir = $env:SYSTEM_DEFAULTWORKINGDIRECTORY
Write-Host "Source directory: $sourcesDir"
$repoDir = Join-Path -Path "$sourcesDir" -ChildPath "MyRepo"
Write-Host "Target directory: $repoDir"

$url = "https://dev.azure.com/my_org/my_team_project/_git/MyRepo"

# Since using PAT, don't need real username, just non-empty username. Use "x".
$username = "x"
$pat = $env:PAT # passed into inline script as environment variable
$credentials = ("{0}:{1}" -f  $username, $pat)
$urlWithCreds = $url -Replace "://", ("://{0}@" -f $credentials)
Write-Host "Clone URL: $urlWithCreds"

git --version
git clone --progress $urlWithCreds
Set-Location "$repoDir"
git checkout master
git log -1 # show the latest commit

You will have to create a PAT with code permissions and pass it into the script as an environment variable. This task will clone the repository to a directory of the same name within your existing repo.

Putting it in a task group allows me to manage it in one place, and then all my more complicated build scripts can be under version control.

There is an active feature request for the capability to source from multiple repos. It looks like it is in the roadmap: https://developercommunity.visualstudio.com/idea/365522/allow-tfs-build-to-depend-on-multiple-repositories.html


backup dockerise mysql database from host volume

I have a little script for my MariaDb database inside a docker container.
The setup is comparable. I have a database container with a mounted volume. Furthermore I have a second volume for backups mounted under /backup.
The script is run on the host via cron and uses the database instance inside the container, so only one database accesses the data. That is the docker exec part.
I'm sure, you can adopt it to mysql. You only have to configure the CONTAINER_NAME and change the command inside the container.
#!/bin/bash
DATE_TIME=`date +"%Y_%m_%d_%H_%M"`
CONTAINER_NAME="mariadb"
CONTAINER=`docker ps -q -f name=${CONTAINER_NAME}`
PASSWORD=`docker exec ${CONTAINER} sh -c 'echo $MYSQL_ROOT_PASSWORD'`
BACKUPFILE=${DATE_TIME}_${CONTAINER_NAME}.backup.gz

echo "Backup of MariaDb"
echo ""
echo "Old backps will be deleted."
echo "Datenbank Container Name: $CONTAINER_NAME"
echo "Datenbank Container:  $CONTAINER"

echo "Writing backup to ${BACKUPFILE}"
echo ""
docker exec ${CONTAINER} sh -c "find /backup/ -mindepth 1 -mtime +8 -delete; /usr/bin/mariabackup --backup --user root --password=${PASSWORD} --stream=xbstream | gzip > /backup/${BACKUPFILE}"
echo ""
echo "Backup finished"


How to build a complex parallel Jenkins pipeline?

Based on the comments to my question, and some basic testing the following seems to work:

Use nested parallel statements. (Thanks @lawnmowerlatte)
Although today (May 15th 2017) Jenkins Blue ocean does not visualise this properly. Possible solutions discussed here


If nothing exists to easily mock Consul, have you considered spinning up a Consul service along with your other microservices? Consul can run in Docker, so this should be fairly straightforward if you use docker-compose and your apps can be configured to point to a different Consul endpoint.

Once your docker-compose is configured to spin up all the infrastructure/microservices required for your test, you will be able to use whatever tools you're familiar with to test your microservices, e.g. Selenium.


Build Docker images with DinD and BuildKit

Caching of layers with buildkit in an external registry requires an extra step or two depending on how you want to cache your layers. The easy option is to include a build arg that enables the inline cache:

docker build --build-arg BUILDKIT_INLINE_CACHE=1 --cache-from ${LATEST} -t ${LATEST} -t ${IMAGE_COMMIT_TAG} .

Note that the inline cache only caches layers for the target stage that was pushed in the image, so other states in a multi-stage build would need to be built and cached separately, or rebuilt without caching, neither of which is ideal.

You can also cache to a local file, or push the cache to a different registry image rather than inline with the image you pushed. Unfortunately the standard docker build CLI doesn't have access to all the buildkit flags to enable this. Instead, you can install buildkit directly or use buildx which is a CLI plugin for managing buildkit. You can install buildx separately but in current releases it's available with experimental CLI options that can be enabled with export DOCKER_CLI_EXPERIMENTAL=enabled. I believe you'd need to create a container based builder for all of the buildkit options, which can be done with:

docker buildx create --use --driver docker-container --name local ${DOCKER_HOST:-unix:///var/run/docker.sock}

Then you should be able to run something like:

docker buildx build --cache-from ${IMG_CACHE} --cache-to ${IMG_CACHE} -t ${LATEST} -t ${IMAGE_COMMIT_TAG} .

More documentation on the buildkit caching is available at: https://github.com/moby/buildkit#export-cache

More details on buildx is available at: https://github.com/docker/buildx


How can we prepare GitLab runners using Ansible deploys

A gitlab runner is a process that f.e. tests, compiles your project, pushes the artefact to you artefact store etc.
A Gitlab runner can be tagged inside Gitlab or via config.toml - so - you can assign runners via tags to Gitlab projects (shared).
Gitlab runner are not designed to run an application 24x7. So - you assign a project to a runner (or a tag or all) to build you application). If you want to deploy the project, use the K8S deployment inside Gitlab (that must be connected to a specific Kubernetes environment based on you Git-branches etc). Or (just an example) run an AWX task that is able to push your new artefact via the artefact storage (docker registry). Also you could just call a SSH remote command on a script (via Gitlab runner) to start a deployment - but don't use the runner to "host" your environment. They will stop (per default after 1h) without a return code.


Is it possible to define a minimum code coverage using Pygradle?

At the moment it looks like it is not possible to define a minimum code coverage as the ticket that was created is still open https://github.com/linkedin/pygradle/issues/184


The service can be always-on listening on the queue for jobs, or single-job started by the queue only when new work arrives.
The tradeoff is startup time for single-jobs vs cost for always-on service.
Define the requirements - number of jobs per day, max latency, ram/cpu needed, etc and then select the right technology that can deliver it in the most cost effective way.
From AWS Lambda, AWS Step Functions, through K8S Jobs and AWS Fargate to AWS EC2 on spot pricing, and more. Plenty of options, depends on the requirements.
Hope that helps :)


Jenkins Job and Jenkins Pipeline are basically the same. In a pipeline you define the steps of your job as groovy code (actually it is CPS https://github.com/jenkinsci/workflow-cps-plugin, but that should in general just be a custom groovy interpreter).

The point, that is making pipelines "better", form my perspective, is, that you can add those in so-called Jenkinsfiles alongside your code. So you have your build job versionized alongside your application code.

So I would recommend: Take the time to learn how these Jenkins pipelines work and use those. But if you do not want or if you do not have the time, then doing it with plain-old Jenkins jobs is fine as well.


docker swarm init made docker go corrupted

This response is for
Kernel: Linux 4.4.132-1075-rockchip-ayufan-ga83beded8524
Architecture: aarch64 aarch64 aarch64 GNU/Linux

I had the same issue when issuing commands to a cluster in swarm mode.

Running docker info would result in Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

Running sudo systemctl start docker would result in a failed service state and output reading

Job for docker.service failed because the control process exited with error code.
See "systemctl status docker.service" and "journalctl -xe" for details.

Running systemctl status docker.service would result in

docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Thu 2019-05-16 06:03:46 UTC; 1min 12s ago
     Docs: https://docs.docker.com
  Process: 3874 ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock (cod
 Main PID: 3874 (code=exited, status=2)

Running journalctl -xe wouldn't result in any information related to docker

Running sudo dockerd -D would result in lots of output with a certain line reading
error="name conflicts with an existing object" module=node node.id=105rworelt1fvw9ttjhnf61aw

Looking in /var/lib/docker there is a folder called swarm and doing a grep on all files and folders in the swarm directory for the
string 105rworelt1fvw9ttjhnf61aw came up with two files with the string 105rworelt1fvw9ttjhnf61aw in them.
sudo grep -r "105rworelt1fvw9ttjhnf61aw" /var/lib/docker/swarm/

Those two files were /var/lib/docker/swarm/.tmp-state.json504201306 and /var/lib/docker/swarm/state.json

At this point I wasn't feeling very surgical so I removed all files and folders under the /var/lib/docker/swarm and then tried sudo systemctl start docker again

A bit surgical: sudo rm -ri /var/lib/docker/swarm/

Note: Don't delete the swarm directory. If you do just create it again.

After this a docker info would work and I was able to docker swarm init again and deploy a stack without issues


How to storing encrypted secrets in code for a serverless.com project?

There are a couple of elements to consider here:

It is valuable to have the capability to maintain configuration with a separate release cadence from the code that is being configured. Doing so encourages rotation of credentials in an automated manner, do it regularly make it painless.
It is entirely reasonable from an "Infrastructure as Code" point of view to store all configuration in Git, albeit probably in a different repository as the scope of configuration is likely to span more than one service.
Assuming both of the above statements are relevant to your situation, I would be inclined to encrypt the data with KMS and store it in S3 or DynamoDB.  I would even go so far to say that in very simple situations it might be suitable to encrypt and store the data within KMS itself.

There are several open source libraries that will support this operation:

Python: CredS3
Go: Sneaker


Context of DevOps in ISO 27001 security audits

In my experience auditors tend to follow a Top-down Risk Assessment process whereby they review the inherent risk to business and IT for a company or system, then look at the controls that company has in place to mitigate the inherent risk and the residual risk that remains.

A top-down risk assessment opposes the way most technical people work, in that we look at specific vulnerabilities, defects and design flaws and then work up from there in a more bottom-up approach.

There is nothing inherently wrong with either approach they both have a place in any given company, so in answer to your questions:

Auditors do not tend to ask domain-specific questions such as Have you checked your API for OWASP breaches; they do tend to ask questions such as What controls are in place to mitigate the risk of data loss due to software vulnerabilities?
The thing to realise about ISO 27001, and most standards, is that they are not prescriptive - they are designed to provide a standard business language to enable communication between Business/IT and Auditors.  More often than not "implementing" any standard is about identifying the existing processes that meet the requirements of the standard and documenting them, then looking for any gaps or flaws and plugging the holes.

The same logic goes for ISO 9000, ITIL, COBIT, COSO, etc. it's very rarely about enforcing some pre-defined process. Instead, it is a case of creating a mapping between what the auditor is looking for and what the business is doing.


Can I run a Ubuntu 16 Docker image on a Ubuntu 18 host? Will I need to install hardware drivers already available to the host?

Docker containers do not use hardware drivers. A containers network connection is abstracted by Docker bridge.

I highly recommend you seek some free tutorial videos or check out Linux Academy to better understand Docker. I promise it will give you great insights.


Azure DevOps Different Post Deployment script per environment

You can define multiple stages in a release pipeline. Each stage (i.e. Dev, Prod) can have its own set of tasks and/or other factors that differ. If you want the same set of tasks, you can duplicate a stage then change what's needed. Alternatively, you can configure the tasks using pipeline variables. The variables can be defined to have different values based on the stage scope (or applicable to the release scope). This way, each stage is essentially the same set of tasks but at runtime each behaves differently based on the variable values applicable to it at runtime.
See the MS docs on stages based on your pipeline type preference:

YAML
Classic


Is there a way to export details of all EC2s in an AWS account?

You can use AWS CLI and run the below command to get details of all ec2 instances in your account.
aws ec2 describe-instances

You can also use filters, if required. Please refer the documentation.


Sonarqube taking massive amounts of memory

Please refer to the following article on the SonarQube - SonarScanner setup best practices. It will be good if the following hardware recommendations  are considered for the SonarQube installations. Following are some of the important considerations.

SonarQube server should be on a real DB not on H2.
The analysis should occur very close to the SonarQube DB.


Is there any Selenium-like software for executables?

I'm not a Windows user, but I'd start at Comparison of GUI testing tools, looking for those supporting Windows for both testing and tested system.


Calling a Terraform object in seperate folder

Using a remote state file and referencing the state from a different state template would be the way I would go. Would need to add an output to resource that needs to be shared as well as configuring remote state storage, access controls, and security.
This URL has example of setting up the remote backend in azureRM.
https://www.terraform.io/docs/backends/types/azurerm.html


kubectl create deployment not has the option containerPort

From the kubernetes docs I see this example:

$ echo 'apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: deployment-example
spec:
  replicas: 3
  revisionHistoryLimit: 10
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.11
        ports:
        - containerPort: 80
' | kubectl replace -f -

So under your containers entry you should be able to add:

        ports:
        - containerPort: 80

Is that what you had in mind or, at least, good enough?  You seem to want this as a command line option for the kubectl command and I don't see anything that indicates this is possible.


Migrating data directory between major db version can be problematic.

I can suggest to create and restore db dump.
You can upgrade postgresql db container as follow:

Backup database
Shutdown postgres 11 container
Run new postgres 12 container
Restore backup

or single line command:

docker exec postgres-11-container pg_dumpall -U postgres | docker exec -i postgres-12-container psql -U postgres


One NAT attachment for two private subnets

Remember that you should be engineering to your requirements around cost, resilience and performance.

If your constraint is around cost, then yes, a single NAT gateway, with both subnets having routing tables that route public IP traffic to the NAT gateway is a viable approach.  If you are this cost sensitive, are you sure that AWS is the right hosting environment?

If your constraint is around having all traffic arrive on the internet from a single elastic IP address, then may i suggest that having two NAT gateways in different subnets in different AZ's is a better choice. You will need one elastic IP address on the active NAT gateway, with routing tables for the private subnets pointing internet traffic to the active NAT gateway.  In the event of a network failure, have a script that moves the elastic IP address to the standby NAT, and then update the route tables to send internet traffic to the second NAT, making it now the active NAT.

Do not fall into the false economy of thinking that you can spin up the replacement NAT at failover time, as it only takes a few seconds. When large catastrophic failures happen within AWS, the processes for the creation of new resources into the surviving AZ's become far busier than normal, and new requests seem to be queued. you would have an unpredictable RTO with this approach. A single NAT should be treated as accepting that on rare occasions, a 12 hour outage is acceptable.


How to automatically update Swarm services when the Docker daemon restarts

You don't need a restart policy with swarm mode. Swarm mode will constantly try to reconcile any differences between the current and target state. I've seen cases where the restart policy enforced by the docker engine, and the implicit reconciliation in swarm mode have conflicted and resulted in issues. The restart policy would also restart the stopped container which would implicitly break the goal to use a new image since the image can not change for the life of a container. Swarm mode, conversely, will recreate any stopped containers with a new container.

To have swarm use the current value of a tag, rather than resolving that tag to an image digest on the initial deploy, you need docker stack deploy --resolve-image .... Note that I haven't tested if this will pull new images, I've only needed to use the feature to use locally built images that shouldn't be pulled from a registry. If Swarm mode doesn't pull new images, you'll also want an extra process to pull your image periodically.


you can change the trigger working process and rewrite and re schedule the process. Which the way you want.
it will work.

Hope it helps.


How to deal with docker compose's naming convention of named volumes?

Docker Prepends the current folder name with all the components name created using docker compose file.

Eg : If the current folder name containing the docker-compose.yml file is test, all the volumes,network and container names will get test appended to it. In order to solve the problem people earlier proposed the idea of using -p flag with docker-compose command but the solution is not the most feasible one as a project name is required just after the -p attribute. The project name then gets appended to all the components created using docker compose file.

The Solution to the above problem is using the name property as in below.

volumes:
  data:
    driver: local
    name: mongodata

networks:
  internal-network:
    driver: bridge
    name: frontend-network

This volume can be referred in the service section as

services:
  mongo-database:
      volumes:
        - data:/data/db
      networks:
        - internal-network

The above name attribute will prevent docker-compose to prepend folder name.

Note : For the container name one could use the property container_name

services:
  mongo-database:
    container_name: mongo


Set environment variables then run script in Jenkins Scripted Pipeline

I figured it out. I was cloning directly into the workspace and then setting my environment variables to point to the workspace as well. I modified both those things. I now create a dir in my workspace and clone into it and I also set my environment variables to directories inside my workspace. Like so:

node('build-01') {
    withEnv(["CMAKE_INSTALL_DIR=${WORKSPACE}/cmake_install", "SDK_INSTALL_DIR=${WORKSPACE}/sdk"]){
        stage('Building') {
            echo "[*] Starting build (id: ${env.BUILD_ID}) on ${env.JENKINS_URL}"
            try {
                sh 'ls -l'
                //ls shows the damn file
                dir('path/to/checkout') {
                    sh '. ./setup-target'
                }
            } catch(all) {
                sh "echo 'Failed to run setup-target script with error: ' ${all}"
            }
        }
    }
}

This works.

Note: Since this site is in beta phase and I barely got any views I posted the same question to stackoverflow, here it is for refernce: https://stackoverflow.com/questions/56015463/set-environment-variables-then-run-script-in-jenkins-scripted-pipeline


Q: Is it possible to share variables across instances?

A: Yes. The dictionary hostvars keeps the variables. Quoting from Accessing information about other hosts with magic variables

  hostvars lets you access variables for another host, including facts that have been gathered about that host. You can access host variables at any point in a playbook. ...

{{ hostvars['test.example.com']['ansible_facts']['distribution'] }}

For details see Scoping variables and Caching facts.

With “Fact Caching” disabled, to share information among Ansible playbooks, it's possible to store all hostvars in a file. For example with this template

  $ cat my_hostvars.json.j2
  my_hostvars_all:
  {% for my_host in ansible_play_hosts_all %}
    {{ my_host }}:
      {{ hostvars[my_host]|to_nice_json }}
  {% endfor %}

the playbook below stores hostvars of all hosts in the dictionary
my_hostvars_all and put it into the file
{{ inventory_dir }}/my_hostvars.json at localhost(master)

  - hosts: test_jails
    tasks:
      - set_fact:
          test_var: "test_var_in_{{ inventory_hostname }}"
      - template:
          src: my_hostvars.json.j2
          dest: "{{ inventory_dir }}/my_hostvars.json"
        delegate_to: localhost
        run_once: true

The dictionary can be included in the next playbook. For example the playbook below

  - hosts: test_jails
    tasks:
      - include_vars: my_hostvars.json
      - set_fact:
          my_hostvars: "{{ my_hostvars_all[inventory_hostname] }}"
      - debug:
          var: my_hostvars.test_var

gives

  ok: [test_01] => {
      "my_hostvars.test_var": "test_var_in_test_01"
  }
  ok: [test_02] => {
      "my_hostvars.test_var": "test_var_in_test_02"
  }
  ok: [test_03] => {
      "my_hostvars.test_var": "test_var_in_test_03"
  }


Don't worry about putting each stack in its own set of subnets, that's an overkill.

Simply make the resources' Security Groups refer to other SGs in your stack templates and not to subnet CIDRs.

What that means is for example the RDS SG sg-123456 will permit traffic from App SG sg-9876543 and not from the App subnet 10.20.30.0/24 you'll be fine from the network isolation perspective.

We usually have the VPC infrastructure shared for dev, test and feature stacks. That's perfectly fine. Moreover we also use CloudFormation Exports to simplify the templates.

Hope that helps :)


How to connect a Bamboo remote agent from within Intranet

So I am assuming that

1) you have a PC on your LAN.  The PC can only talk to port 80 outgoing via a proxy

2) you have a bamboo server running a cloud service

3) you wish to initiate a connection from the bamboo server to the PC

Don't get your hopes up but this answer https://stackoverflow.com/questions/19161960/connect-with-ssh-through-a-proxy

explains how to use a http proxy to tunnel ssh.  Unfortunately most http proxies disallow the CONNECT method

Once you have the ssh to the remote cloud service set up then try it again
with a -R option to allow a reverse tunnel

Then you should be able to connect via the reverse tunnel back to the Lan PC


Have you thought about what happen with stale/old data??

Each new version creates a ton of new entries.
Management becomes a nightmare
the DB becomes a SPOF(single point of failure).

Better Solution: Service discovery
The better solution would be to use a Service discovery System and annote each Service with their version and the environment as a Metadata tag. This keeps the System more manageble.
More Help
IMHO, sticking Linux Code into a Docker Image and run multiple copies of it, is the best way of getting a immutable infrastructure up and running.


Secure way of uploading data to Google Cloud Storage

Yes, your approach sounds okay to me. I am not sure how you "ship" machines but it is a good idea to use a service account per machine as you can delete the service account key if the machine ever becomes compromised.
You can, and should, fully automate the process of generating the credentials key files using the Google APIs or gcloud cli.
It is best practice to periodically rotate (re-issue) credentials key files. This is in case they are accidentally leaked in a way that you do not notice. If you always revoke the key and issue a new one every month then it limits the total damage somewhat.
If you do rotate keys then you have a key distribution problem. How do you securely distribute the new credentials key files onto machines you shipped in the past? You can use GnuPG to create a public-private key pair on every host. Each month you can run a script that generates a new credentials key file per machine, encrypt it with the machine's public GPG key, then delete the old service account keys.
The script that pushes images to a bucket must then periodically "phone home" to collect a fresh GPG encrypted credentials key file. It can then decrypt it with its own private key. One way to do this is to put the encrypted keys into a different bucket. Create a single service account that can only read that bucket. Create a single credentials key file for that service account that you install onto every machine. That key can then only be used to only collect encrypted keys that can only be decrypted on the correct host.


Automated Docker swarm reinitialization and token distribution

Have the Swarm Manager place the worker join token in a Key/Value store.  We use AWS Parameter Store but I believe Consul would be another option (I don't use it).

Each node can then retrieve the token while it is bootstrapping.


How can a Sys. Admin skill up to work as a DevOps engineer?

Have a look for DevOps Engineer roles and check out the list of technologies that they are seeking. Typically you'll need automation scripting skills (like Bash, Python or PowerShell), configuration management skills (like Chef, Puppet or Ansible) and some fundamental experience maintaining and configuring a Build/Release pipeline. For this, look to get experience with CI/CD tools like AppVeyor (.NET), Jenkins, Travis CI etc. You should also be familiar with Git and source control systems in general such as BitBucket or TFS. You would also benefit from becoming familiar with automated testing techniques.

If you haven't already, consider hosting some open source products in Github and setting up a pipeline for them that performs automated tests on commits as well as (if possible) publishing to a package management system. This is great experience as well as demonstrates your skills.

For a more general understanding of the philosophy of DevOps (as beware that it's a culture as well as a technology set) do as much reading as you can on the topic and start following (e.g via Twitter) the likes of Gene Kim, Jez Humble, and Patrick Debois. Also I recommend you read The DevOps Handbook which they authored. It is a very thorough explanation of what DevOps is as well as how organisations might adopt it.

You could also check out DevOps related Meet Ups and Conferences (such as Velocity, DevOpsDays and Config Management Camp). The opportunities will of course vary depending on where in the world you are.


Internet connection inside Docker container in Kubernetes

In my opinion k8s should not be used to build docker images as it is an orchestration platform, but there is an open issue in the GitHub k8s repository. In conclusion, building docker images on a k8s cluster seems not to be possible at the moment.


Problem in integrating Travis-CI with core PHP projects

You need to write some unit tests and set the script variable in your .travis.yml to actually run a file.

By default, Travis CI runs the command phpunit without any arguments. When this happens, phpunit doesn't know what you're asking it to do, and shows a help message, then exits with error code 2 (i.e. non-zero, which indicates an error occurred).

The Beginner's Guide shows you a possible solution of setting the script variable to run a Test.php file with your tests in (insert this into your .travis.yml):

script: phpunit Test.php

There is an example of a valid Test.php in the example repo that Travis provides:

<?php
    class Test extends PHPUnit_Framework_TestCase
    {
        public function testOnePlusOne() {
            $this->assertEquals(1+1,1);
        }
    }
?>


Chef remote_file set maximum number of re-directions to zero

Found out that the remote_file resource couldn't handle redirects. Hence I had to write a ruby_block resource that makes use of 'Down' gem.

ruby_block 'download_openvpn_zip' do
    block do
        attempt = 2
        begin
            retries ||= 0
            tempfile = Down::NetHttp.download("http://www.someurl.com/#{zip},max_redirects: 0)
            FileUtils.mv tempfile.path, "#{node['openvpn-conf-path']}/#{tempfile.original_filename}"
        rescue Down::TooManyRedirects => e
            puts "\n \t ERROR: #{e.message}"
            retry if (retries += 1) < 1
        end
    end
    action :run
    notifies :run, 'execute[unzip_file]', :delayed
end


Running additional commands on Docker image startup

Simply write a Dockerfile which uses that image as a base. Then you can replace individual files (like the entrypoint) easily.

Also, instead of replacing, you could rename the original entry script, create your own, which does whatever it needs to do, and then just calls the original.

Somewhat like this:

Dockerfile:

FROM fancy-base-image:1.2.3
RUN mv docker-entrypoint.sh docker-entrypoint-original.sh
ADD docker-entrypoint.sh

docker-entrypoint.sh:

#!/bin/bash
echo Do something.
./docker-entrypoint-original.sh
echo Do something else

Obviously this is just a fake example, you'll have to adjust for directory and file names and such.


What is the difference between dockerd and com.docker.service in Docker for Windows?

dockerd is a persistent process that manages containers. Docker uses different binaries for the daemon and client. ( from Docker site ). Docker Daemon is a server which interacts with the operating system and performs all kind of services. The Docker Daemon listens for REST API request and performs the operation. Its process running on the background.

Docker service is the utility we use when we run any docker commands e.g. docker run (docker container run), docker images, docker ps etc. It allows us to run these commands which a human can easily understand.


Docker & Django - what directory does localhost point to?

This is most likely a problem with the django configuration better fitting to stackoverflow, and not so much a devops problem.

Check the settings.py for the following entries:

INSTALLED_APPS = [
    (...)
    'django.contrib.staticfiles',
]

At the end of this file

STATIC_ROOT='staticfiles'
STATIC_URL = '/static/'
STATICFILES_DIRS = [
     os.path.join(BASE_DIR, 'static'),
]

source Django manual: How to static-files


Bitbucket Pull Request Builder Jenkins job is not getting invoked

It took me a while to understand this as well as I had experience with Bitbucket Server and not Bitbucket Cloud which is different.

You do not need to enable the webhook for the Bitbucket Pull Request Builder plugin in Jenkins.

Just ensure you set the RepositoryName and RepositoryOwner properly from the URL of your repo as such: https://bitbucket.org/RepositoryOwner/RepositoryName

Once you have that set, a new PR or commit to branch of existing PR will trigger your job.

P.S. There's a bug in the plugin where you can't use git for other repos such as workflow library or pipeline from scm. Git will try to use the same hash to checkout all repos which will fail the build.

P.P.S. The webhook in Bitbucket cloud uses a POST with Json data so that can't be used with the git plugin in Jenkins unless you don't care about the hash or branch.

EDIT:
One last note, I've since moved to use the multibranch pipeline with bitbucket branch plugin and that works great with the webhook as well as allows us to use a workflowlib library


Running multiple websites on Docker in isolated or shared database containers?

Docker and all the other fancy names are just tools. The examples are just examples.

Resource-wise, Docker changes nothing much beyond usual Linx/Unix multiprocessing. It does not have the overhead of VMs, and little overhead over just starting the process directly ( https://stackoverflow.com/questions/21889053/what-is-the-runtime-performance-cost-of-a-docker-container ).

So it is really up to you to decide how to use it. If you have closely related applications under your own command, maybe in your intranet, why not use a shared DB. If they are customer DBs, or you might need to run different versions of the software, or different incompatible configurations, then separate services might be fine.

For RDBMSses especially, they are usually very good at managing resources (RAM, HDD) and so for this case specifically, running only one may just be the best thing. It's really more about what your needs are than general guidelines.


how to checkout only specific folder from git repo((monorepo)) and build

You can check out specific files or folders from a repo using git archive.  I've done this from a Jenkinsfile using a shell command like so:

sh("git archive --remote=${repo_url} ${ref} ${path} | tar x")

You will, of course, need to set the variables repo_url (the url of the remote repository), ref (the ref/branch/commit/etc. to check out), and path (the path of the file or folder within the repo to check out).


What are best practices for implementing SSL on Jenkins?

Besides the transport layer you should first of all enable
Access Control

to protect jenkins from public access.


This has worked - the following query has to return content-type application/xml:

 /solr/COLLECTION/select?q=*:*&rows=0


Just to clarify if I got you right.
Do you want to have a separate service for every pod/replica in your statefulSet?
If so - this is not how K8s sees services being used. The service is used for grouping and load balancing purposes.
In your case, if you want to have 1-on-1 connection between an external client and a pod, I suggest deploying your statefulSet a number of times with replicas=1 and exposing a service (as NodePort/LB) every time.


Pros/cons of discontinuing a DevOps workflow?

It's not a good idea.

In my experience you'll gain the disadvantages of both while any projected advantages will somehow fail to materialize.

Itemized:

You will lose speed.
IT will comply with their own standard. The new task (for them) will follow the same 'sluggish' template all their work now has. Be prepared they will find it challenging - so even less speed than standard simple actions.
You will fail to offload.
IT will lean on you guys for every single anomaly. You will put in effort to get one guy up to speed -  and the next thing you now you'll be repeating yourself because the following task/problem/day there will be a new guy, again.
Documentation will be required, but it will not help.
Again template behaviour will be that short manuals will fail to catch every anomaly, and thorough texts will not be read for being too long. So any investment here will be a loss, likewise the enormous effort needed to implement improvements to get your tools to 'shrink-wrapped'quality.

Last but not least any problems will reflect on you guys. Tar, tarbrush principle.

If the above sounds cynical, well, I'm afraid I've been there. Repeatedly.

What to do instead?

Go shopping in the IT department, find yourself a useful candidate, and keep this guy 'on loan' to relieve you workload.


How to Block Traffic From a Single IP in AWS

Yes, we can block traffic from single IP or IP range. Please follow below steps to do this

Open VPC dashboard

Open the Network ACLs view

Open the ACL editor

Select the subnet to which your EC2 instances or load balancers are connected.

Click Inbound Rules

Click Edit

Add a rule to block the traffic/IP

You will now see the ACL editor. On the last row, you can add a new rule.

Here is how you should fill out the fields:
Rule
Use any number less than 100, which is the number of the default accept-all rule. This is important because rules are evaluated in order, and your rule needs to come before the default.
Type
Select “All traffic” or Particular Protocol which you want to Block
Source
The CIDR you want to block. To match a single IP address, enter it here and append /32. For example, I blocked 22.87.45.187/32
Allow/Deny
Select “DENY”
Now click Save and you should see the updated rules table.
Fore more details visit https://www.serverkaka.com/2018/05/block-traffic-from-single-ip-in-aws.html

UPDATE: How to do it in CLI

NOTE: This example command includes --dry-run option, so it is safe to run. It will not create NACL rule. Run it without this option, if you want to apply it.

aws ec2 create-network-acl-entry --dry-run --network-acl-id acl-146adb6f --ingress --rule-number 20 --protocol tcp --port-range From=80,To=80 --cidr-block 22.87.45.187/32 --rule-action deny

More examples:
https://docs.aws.amazon.com/cli/latest/reference/ec2/create-network-acl-entry.html#examples


How to upgrade job-dsl files to Jenkins 2 pipelines?

No, there still isn't any common practice for this since this is not very common.

I would recommend you to create a Development Jenkins Environment and run tests against it. Right now you will have to do that from scratch since there are not many open source repositories that could help you.

So, how does this Jenkins Development Environment look like?

For me it is a docker-compose file that you can run and get everything you need. If you can automate the creation of the jobs it will be a plus.


Jenkins: store build information for later reference

If you want to use a database, you can always shell out to a database:
sh "mysql -u my_username -p my_password -h my_hostname -P 3306 -e \'INSERT INTO my_table (column1, column2, ...) VALUES (value1, value3, ...);\'  my_database"

You can also write out your data to a JSON or YAML or whatever type of file and then store it in Jenkins' built-in artifact storage with archiveArtifacts.


How to implement blue-green deployment with one master branch

It is worth separating your Continuous Integration "Build" from your Continuous Deployment "Pipeline".  For the build portion, CircleCI seems to be your tool of choice, for the latter I recommend using something like AWS CodeDeploy which natively supports Blue-Green Deployments, for completeness you can use CircleCI to orchestrate AWS CodeDeploy:

Separating your pipeline phases has several benefits:

You only need to compile/test once from the master branch; then promote the packages through the Artefact Repository.
Your packages are stored forever in an Artefact Repository, for example Sonartype Nexus or JFrog Artifactory, this means you can reproduce an old build as you have the specific packages used for deployment.
CodeDeploy will handle the release patterns for you without having to think too much about how to achieve Blue-Green Deployments.

It is also worth stating that you can model the whole pipeline in Application Release Automation tools such as BuildMaster or Nolio.  Personally, I prefer to build my pipelines from multiple 3rd party SaaS solutions.


Do I have to include gitlab variables in my terraform code in order for my credentials to be recognized?

The answer to this question is no. I merely forgot to export the AWS_SECRET_ACCESS_KEY variable in the before_script section.


How to install docker in Raspbian Virtual Machine

The Docker website doesn't know about your operating system and has no docker installation for you.

Try the one packaged by raspbian:

sudo apt install docker.io

The name is different in some distributions, you can use:

apt-cache search docker

To see if you can find one. If you can't find any, then switch to an older (or newer?) version of Raspbian.

Or you can search for instructions on how to compile it from source code.


I have sometimes used the pattern where all pipeline things go in a .pipeline/ directory of the app, including the Jenkinsfile. I would, in your case, add several pipelines (ie, Jenkinsfiles) - one per microservice as you propose, and one for the web app.

Then, you define independent multibranch pipeline jobs, based on the location of the pipelines. In other words, you create 5 multibranch pipeline jobs, (one for the web app, one for each microservice), and attach each job to the relevant pipeline file in your repsitory. In this way, they will be tested independently.

The important thing is to properly specify the correct triggers. The microservices will be triggered on changes to code in their path. They may also have as post-build actions the triggering of the web app pipeline, for integration tests, etc.


Control order of container termination in a single pod in Kubernetes

But maybe there is some way.

Order of termination (or start) as such is not something you can easily define, but you can try a workaround with Container Lifecycle Events.

General idea is that on a PreStop handler for proxy container you can check if app container finished working (say localhost:appport is not active or whatever api check you can muster to determine when app pod is down) and then continue with proxy pod shutdown normally.


What are the correct permission settings when running Docker in a Jenkins Pipeline?

So after a debug session in chat what is needed is to allow the user running jenkins to be able to sudo docker passwordless on the docker host.

A typical sudoers file on ubuntu could be in /etc/sudoers.d/jenkins

jenkins_user ALL=(ALL) NOPASSWD:ALL

Be warned this allow jenkins_user to run as root without password any command, a better file should be:

jenkins_user ALL=(ALL) NOPASSWD:/full/path/to/docker
jenkins_user ALL=(ALL) NOPASSWD:<other needed command to be run as root>

This will allow to start the container as root and as such it gives all rights within the container itself by running as uid 0.

Useful resources used aside:

Jenkins docker plugin readme
How to setup passwordless sudo on Linux?


Access private DockerHub repo in Docker Swarm

You need the --with-registry-auth flag, e.g.:

docker stack deploy -c myapp.yml --with-registry-auth myapp

From the docker stack deploy --help output:

--with-registry-auth     Send registry authentication details to Swarm agents


How should CI/CD deploy to production environment - copy artifacts or git pull from the server?

There are lots of bad examples on the Internet.  Doing a git pull to distribute code is fabulous for development, but prone to all sorts of issues in practice.  It should not be used for production deploys without careful consideration of the negative consequences.

What happens when somebody checks out a different branch?  You'll be sad when you keep git pulling and nothing new shows up.
How do you verify that the machine is running the expected release?  You can get this from git, but it is easier with most artifact systems.
How do you release to machines that are on an air-gapped network?  Simulating your git server on a private network will be a fun time for you.

I'm not sure what format npm uses for artifacts, but it is also probably a good idea to back them up off of the CI/CD server and use a dedicated set of machines for serving up the artifacts.  If you spin up 100 machines at once would your CI/CD server handle it or fall over?  Isolating the artifact serving will avoid your CI/CD server being swamped and useless until the deploy is over.


How to implement the immutable server pattern without loosing the ability to do post-mortems?

First of all, removing ssh on an immutable server doesn't guarantee there'll be no change, it's more that as there should be no need to change something you reduce the attack surface by removing a remote access channel.

One way to keep a sort of post-mortem is log centralisation. There's a myriad of methods to achieve it, ELK stack, Splunk, syslog...

Another more crude way to keep a post mortem for an immutable server is to have a script on the shutdown process (an immutable server failing would be shutdown and a new one spin up to replace it) to gather a core dump of the program, a memory dump and send them to a remote system for analysis along with most of the logs.

Main advantage of this solution is that you get back only failing system information at time of problem, allowing to gather larger informations than getting them periodically.

It's hard to be more specific on how to achieve this, each distribution has some way to get things and I've no generic example.


Problem solved, I think: the agent comes with its own libcurl.so.3 in /opt/oms/lib, it doesn't use the system one at all. Once I replaced that, it seems to be succeeding. Previously:

$ cd /opt/microsoft/omsconfig/Scripts
$ ./PerformRequiredConfigurationChecks.py
instance of OMI_Error
{
    OwningEntity=OMI:CIMOM
    MessageID=OMI:MI_Result:1
    Message=cURL failed to perform on this base url: uks-agentservice-prod-1.azure-automation.net with this error message: Stream error in the HTTP/2 framing layer. Make sure cURL and SSL libraries are up to date.
    MessageArguments={}
    PerceivedSeverity=7
    ProbableCause=0
    ProbableCauseDescription=Unknown
    CIMStatusCode=1
    OMI_Code=1
    OMI_Category=0
    OMI_Type=MI
    OMI_ErrorMessage=A general error occurred, not covered by a more specific error code.
}

But now:

$ ./PerformRequiredConfigurationChecks.py
instance of PerformRequiredConfigurationChecks
{
    ReturnValue=0
}


AWS stores whole AMI as a single non-differential snapshot thus they charge for a full size of files in an image.


Deploying lambda conceiving sensitive variables using serverless

As far as I'm aware. There is no way to encrypt the environment variable value in the variables section of the lambda function. And to be fair, even if there was, it wouldn't provide any benefit in terms of keeping the variable secret. If the variable could be hidden/encrypted in the variables sections to be decrypted later, a user could simply execute the lambda function and print the results of the decrypted variable to stdout.

One approach could be to use aws secrets manager to store and retrieve your secrets. You can then assign your lambda function a role to allow access to the secret store, and use AWS SDK to retrieve your value.


How to run a script or a command on EC2 instance via AWS CLI?

To run ipconfig from the AWS Systems Manager Run Command:

$ aws ssm send-command --document-name "AWS-RunPowerShellScript" --instance-ids "<your instance id>" --parameters commands=ipconfig

Note: If you've got the error, consider specifying the right --region.

This assumes you have your AWS credentials and CLI configured properly. See Systems Manager Run Command Walkthrough Using the AWS CLI for more information.

Here is the practical shell command example of sending and getting the command output:

cmdid=$(aws ssm send-command --instance-ids "i-ch3ng3th1s" --document-name "AWS-RunPowerShellScript" --parameters commands=ipconfig --query "Command.CommandId" --output text)
aws ssm list-command-invocations --command-id "$cmdid" --details --query "CommandInvocations[*].CommandPlugins[*].Output[]" --output text


Groovy script to show only selected item

You can use the Jenkins pipeline syntax to get the groovy script.  For example below code will generate radio buttons for active choice parameter. you can generate scripts like this from your jenkins instance at  http://localhost:8080/pipeline-syntax/. More info is here https://jenkins.io/doc/book/pipeline/getting-started/#snippet-generator

def createInputParameters(){
([[$class: 'RebuildSettings', autoRebuild: false, rebuildDisabled: false],
 parameters([[$class: 'ChoiceParameter', choiceType: 'PT_RADIO', description: '',
 filterLength: 1, filterable: false, name: 'Select_snapshot_or_release',
  randomName: 'choice-parameter-1683871426502398',
   script: [$class: 'GroovyScript', fallbackScript: [classpath: [], sandbox: true,
    script: 'return "Error"'],
     script: [classpath: [], sandbox: true,
      script: '''return[
            \'snapshots\',\'release\']''']]]]),
             [$class: 'ThrottleJobProperty', categories: [],
              limitOneJobWithMatchingParams: false, maxConcurrentPerNode: 0,
               maxConcurrentTotal: 0,
               paramsToUseForLimit: '', throttleEnabled: false, throttleOption:
               'project']])
      }


No Such Host: Kubernetes/Docker cannot pull from private k8 registry

It seems your private DNS is misconfigured. Check your host or make sure that docker-registry.default is the a valid hostname.
You can run a temporary Ubuntu pod to check if that hostname is resolving correctly from your cluster, e.g.
$ kubectl run ubuntu -it --image ubuntu --rm=true -- bash
root@shell:/# apt update && DEBIAN_FRONTEND=noninteractive apt --yes install dnsutils
root@shell:/# dig docker-registry.default
docker-registry.default.    IN  A D0.y0u.h4ve.1P?

Additional suggestions:

If you're using a cloud provider (such as Azure), make sure your Private DNS zone is linked with your cluster's virtual network. Read: Connect privately to an Azure container registry using Azure Private Link
If your container registry endpoint isn't correct, edit the deployment manifest and fix the image path.
Test pulling the image from another VM within the same private network.


What are the pros and cons of Azure Artifacts vs. Artifactory?

JFrog Artifactory pros

Artifactory supports 25+ package types, while Azure supports only Maven, npm, NuGet and Python. You can get Docker and Helm with Azure Container Registry but it’s a separate tool.

Artifactory is fully hybrid - available both for on-premise installations and as SaaS - with full parity between the versions.

With Artifactory you can avoid vendor lock-in - not depending just on one cloud vendor for all of your needs. That also means not being limited to any vendor’s stack for your database, access management, etc.
 Proxy any remote repository - not just the official ones (like Maven Central and PyPI.org)

Artifactory is optimized for multi-site operations - Supports any network topology, over multiple, geographically distant sites and data centers, through different options for replication including pull, push, event-based and multi-push replication.

Rich metadata & AQL - Artifactory provides full metadata capabilities for all major package formats for both artifacts and folders – including custom metadata. Users can tag artifacts and folders with searchable properties. Artifactory also collects build information for build artifacts as part of the CI server integration (see below). AQL gives you unprecedented flexibility in how you search for artifacts. It offers a simple way to formulate complex queries that specify any number of search criteria, filters, sorting options and output fields.

Build info - Artifactory stores exhaustive metadata for all build artifacts deployed to it. With this “bill of materials” it is easy to faithfully reproduce a build, and trace the cause of issues to reduce time to resolution (TTR), even if the build is already in production.

Azure Artifacts pros

if your CI is already built on Azure DevOps, the initial set up for Azure Artifacts may be a bit quicker. The basic functionality of hosting artifacts, so that they can be resolved by other builds will work for you.

Disclaimer: I'm a JFrog employee.


How to make one branch public while the others remain private on GitLab?

I'm not an expert in GitLab but I would assume it to be similar to other solutions like it (bitbucket, github).

A simple way to achieve this would be to clone the repo in another private repo instead of doing a separate branch.  In effect this would work similar to having your local repo and the gitlab repo.  Merging a branch and merging with another repo is not much different with git


Planning a backup system using postgress, chef and vault

First chef runs as often as you configure it to do.

Second Chef is a configuration management system and not a crontab or an orchestration system.

Use chef to configure cron (that can not be not an option, try to find out why this is not considered) for your backup using consul-template to retrieve the secret from vault. Don't use Chef to run the backups because of the caveats you already mention.

However if you really want to shoot yourself in the foot, just add a guard on your backup command like this (604800 seconds is roughly 7 days):

not_if { (Time.now() - File.mtime("/path/to/your/backup/file.tgz") < 604800 }


Can't install MongoDB docker on raspberry

This means that the container that you are trying to pull is not available for the architecture, in this case it would be linux/arm64. Maybe there is an image available now. I would search for 'docker mongodb arm64' to see. I see support for mongodb under arm64v8 architecture, which would be in Raspberry Pi 4. Maybe you want to upgrade, as your chip is arm64v7.

This issue explains error message in a bit more detail.

Answers under this issue are going to help you run MongoDB under ARM64.


Check the image history of your two images with docker image history. Pay careful attention to the sha256 checksum of the files being added on the ADD line. Docker performs a checksum on the files, including some of the metadata on these files, so a single byte change or a permission update will result in a cache miss.

Once you've checked this, if it is not your issue, please update the question with the pull command, the build command and full output, and the history command on both images.

Side note: you likely want to use COPY instead of ADD unless you are pulling remote files or extracting a tar file into a directory.