How to check if pipeline parameter is empty

While creating the credentials parameter in jenkins job, you can specify required: true, then jenkins should validate the credentials paramter.

parameters {
    credentials(name: 'GPG_PASSPHRASE', defaultValue: '', credentialType: "Username with password", required: true )
}

You can also check the details mentioned in the link https://docwhat.org/jenkins-user-credentials

Adding Parameter

/* EXAMPLE */
parameters {
    credentials(
        credentialType: 'com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl',
        defaultValue: '',
        description: 'The credentials needed to deploy.',
        name: 'deployCredentialsId',
        required: true
    )
}

Using in the stage

/* EXAMPLE */
steps {
    withCredentials([usernamePassword(
        credentialsId: '${deployCredentialsId}',
        usernameVariable: 'DEPLOY_USERNAME',
        passwordVariable: 'DEPLOY_PASSWORD',
    )]) {
        sh './my-command.bash --username="${DEPLOY_USERNAME}" --password="${DEPLOY_PASSWORD}"'
    }
}

If you just want to check if the variable is empty or not, you can check isEmpty() in groovy script. Your code can be as follows:

stage ("create bundle"){
        steps{
            script{
                    if ( GPG_PASSPHRASE.isEmpty() ) {
                             GPG_PASSPHRASE = 'custom_string'
                     }
                    amd_distribution_create_bundle credential_id: params.DISTRIBUTION_CREDENTIAL_ID, distribution_url: params.DISTRIBUTION_URL, gps_credential_id: params.GPG_PASSPHRASE, bundle_name: params.BUNDLE_NAME, bundle_version: BUNDLE_VERSION
            }
        }
    }


How well can agile pair programming work in interdisciplinary DevOps contexts?

Agile fits very well with a DevOps culture and pair programming is no exception.

The following are some examples of "pair programming" principles applied in a DevOps context:

A pair of individuals working together on a Dockerfile for their service.
A pairing sits together to work on defining their infrastructure as code.
A traditional developer and an IT administrator building a CI/CD pipeline together.
A specialist and tester sit together to define what the testing environment should look like.
A specialist and a customer sitting together and working on a deployment to the customers infrastructure.


Health monitoring of web pages without using GET check

I see two way to solve this problem:

Check your logs often and ensure there was an access with code 200 in the last N seconds/minutes and no code 5xx meaning there's a server side error. (the often should be coherent with how long you are ok with no entries)
Keep an active check but:

use a HEAD request (so there's less data to return by the server)
use a custom user agent for the check
configure your log exporter to ignore this user agent.
take a larger check interval, 1 second is probably over obsessive

Depending on how often your sites are called you may wish to mix both methods to have a proper check even if there's no traffic from real clients.


How to create Golang Linux binaries using a Windows host

You can easily set the target operating system and processor architecture using the environment variables GOOS and GOARCH respectively. So, as you want to build it for linux operating system, following command with above environment variables will do,

$ GOOS=linux GOARCH=amd64 go build -o hello main.go

Here is the list of all the supported operating system with which you can easily do cross compilation using Go lang.


How do I list pod name/version across multiple clusters in Kubernetes

The best I have found so far for part 2 is to use the JSON output of get pods:

kubectl get pods -o json | jq '.items[] | select(.metadata.labels."app.kubernetes.io/name") | { name: .metadata.labels."app.kubernetes.io/name", version: .metadata.labels."app.kubernetes.io/version" }'

produces

{
  "name": "db",
  "version": "0.0.1"
}
{
  "name": "app",
  "version": "0.0.1"
}


Docker NGINX issue -- unable to edit nginx.conf

I suggest you mount your conf file as a volume:

docker run -v nginx.conf:/etc/nginx/nginx.conf ....

This way you can easily change the file outside the container and then just restart the container. If you change your config file inside the container and then you should have to restart nginx to pick-up the changes. At that point your container will stop because you stopped the main process.

Your local nginx.conf should already have the initial configuration which you can copy from inside the container (cat /etc/nginx/nginx.conf).


In ansible how to select specific tasks in a role according to the playbook or a more general parameter?

Different approaches can be useful for different setups. You can use tags in some cases, or conditional includes (as you describe) in other cases.

  On the other hand, it generates quite a payload of “skipped” tasks in ansible logs, which raises a red sign to me.

You should get familiar with difference between static and dynamic includes in Ansible.

So if you use static includes, you will get a lot of skipped tasks, because every task in every file is added to execution queue but most of them are skipped because of when statement.

tasks/main.yml of your role with actions install/uninstall:

- include: install.yml
  when: action == 'install'
- include: uninstall.yml
  when: action == 'uninstall'

But if you use dynamic includes, only required tasks are added to execution queue and skipped message is displayed for skipped includes.

tasks/main.yml:

- include_tasks: install.yml
  when: action == 'install'
- include_tasks: uninstall.yml
  when: action == 'uninstall'

More on that, you can avoid skipped message altogether like this:

tasks/main.yml:

- include_tasks: '{{ action }}.yml'

The drawback of dynamic includes is that you can't do complete static analysis of your playbook (like syntax checks, etc.) because many things are know only in runtime.


Prometheus - join series based on the key part of a label

label_values(kube_node_role, role)

sum(kube_pod_container_resource_requests_cpu_cores
  * on (node) group_left (role) kube_node_role{role="$node_role"})

kube_node_role out of kube-state-metrics can give you a relationship to join


what is better between Jenkins & AWS CodeDeploy for CI/CD

Both are excellent tools, but it depends on your use cases for which is better. I would highly recommend using the entire AWS Ci/Cd suite (CodeBuild/CodePipeline/CodeDeploy) to gain the full benefit of using your resources in AWS. Here are pros for both:

Jenkins

Multitude of customizability plugins
Integration with all cloud providers (GCP, Azure, etc.)
Open sourced
Configuration as code through Jenkins Pipelines
Able to manage your own Jenkins Server (vs AWS which manages it for you)
Customizable GUI

AWS CI/CD

No plugins (no dependency hell)
Trivial integration with IAM (for both users and infrastructure)
Easy integration with AWS Secrets Manager, KMS, and other secret management tools
AWS managed infrastructure
Infrastructure as code through CloudFormation
Scalable out of the box

TLDR; Choose Jenkins if you want more control/configuration of your deployment infrastructure or you are integrating with many non-aws resources, otherwise, choose the AWS CI/CD suite for less overhead and better integration with AWS.


How can I check if there is a new build the same jenkins pipeline

This should be fairly straightforward to do in a script block.  I don't think there's a declarative-native way to do this.
I use this script block to cancel builds for earlier (i.e. outdated) revisions of PRs:
script {
  // currentBuild is a Pipeline-builtin variable for accessing the Build object
  // https://www.jenkins.io/doc/book/pipeline/getting-started/#global-variable-reference
  def build = currentBuild

  while(true) {
    build = build.getPreviousBuild()

    // exit the loop if we have finished iterating over builds
    // (groovy doesn't have a do...while feature)
    if (build == null) break

    def rawBuild = build.rawBuild

    if (rawBuild.isBuilding()) {
      // abort the build
      rawBuild.doTerm()
    }
  }
}

As you can hopefully see, this iterates over all previous builds and uses the isBuilding() method to find ones that are currently running.
If you want to check later builds instead of previous ones, you can use getNextBuild() instead (I believe the API doc is here, it's been a while since I wrote this so this might be the wrong class).
And if you want to check only one directly adjacent build instead of all earlier/later builds, you can just omit the loop and only call getPreviousBuild()/getNextBuild() once.


A docker image is actually a linked list of filesystem layers. Each instruction in a Dockerfile creates a filesystem layer that describes the differences in the filesystem before and after execution of the corresponding instruction. The docker inspect subcommand can be used on a docker image to reveal its nature of being a linked list of filesystem layers.

The number of layers used in an image is important

when pushing or pulling images, as it affects the number of concurrent uploads or downloads occuring.
when starting a container, as the layers are combined together to produce the filesystem used in the container; the more layers are involved, the worse the performance is, but the different filesystem backends are affected differently by this.

This has several consequences for how images should be built. The first and most important advice I can give is:

  Advice #1 Make sure that the build steps where your source code is involved comes as late as possible in the Dockerfile and are not tied to previous commands using a && or a ;.

The reason for this, is that all the previous steps will be cached and the corresponding layers will not need to be downloaded over and over again. This means faster builds and faster releases, which is probably what you want.  Interestingly enough, it is surprisingly hard to make optimal use of the docker cache.

My second advice is less important but I find it very useful from a maintenance view point:

  Advice #2 Do not write complex commands in the Dockerfile but rather use scripts that are to be copied and executed.

A Dockerfile following this advice would look like

COPY apt_setup.sh /root/
RUN sh -x /root/apt_setup.sh
COPY install_pacakges.sh /root/
RUN sh -x /root/install_packages.sh

and so on.  The advice of binding several commands with && has only a limited scope. It is much easier to write with scripts, where you can use functions, etc. to avoid redundancy or for documentation purposes.

People interested by pre-processors and willing to avoid the small overhead caused by the COPY steps and are actually generating on-the-fly a Dockerfile where the

COPY apt_setup.sh /root/
RUN sh -x /root/apt_setup.sh

sequences are replaced by

RUN base64 --decode … | sh -x

where the … is the base64-encoded version of apt_setup.sh.

My third advice is for people who wants to limit the size and the number of layers at the possible cost of longer builds.

  Advice #3 Use the with-idiom to avoid files present in intermediary layers but not in the resulting filesystem.

A file added by some docker instruction and removed by some later instruction is not present in the resulting filesystem but it is mentioned two times in the docker layers constituting the docker image in construction. Once, with name and full content in the layer resulting from the instruction adding it, and once as a deletion notice in the layer resulting from the instruction removing it.

For instance, assume we temporarily need a C compiler and some image and consider the

# !!! THIS DISPLAYS SOME PROBLEM --- DO NOT USE !!!
RUN apt-get install -y gcc
RUN gcc --version
RUN apt-get --purge autoremove -y gcc

(A more realistic example would build some software with the compiler instead of merely asserting its presence with the --version flag.)

The Dockerfile snippet creates three layers, the first one contains the full gcc suite so that even if it is not present in the final filesystem the corresponding data is still part of the image in same manner and need to be downloaded, uploaded and unpacked whenever the final image is.

The with-idiom is a common form in functional programming to isolate resource ownership and resource releasing from the logic using it. It is easy to transpose this idiom to shell-scripting, and we can rephrase the previous commands as the following script, to be used with COPY & RUN as in Advice #2.

# with_c_compiler SIMPLE-COMMAND
#  Execute SIMPLE-COMMAND in a sub-shell with gcc being available.

with_c_compiler()
(
    set -e
    apt-get install -y gcc
    "$@"
    trap 'apt-get --purge autoremove -y gcc' EXIT
)

with_c_compiler\
    gcc --version

Complex commands can be turned into function so that they can be fed to the with_c_compiler. It is also possible to chain calls of several with_whatever functions, but maybe not very desirable. (Using more esoteric features of the shell, it is certainly possible to make the with_c_compiler accept complex commands, but it is in all aspects preferable to wrap these complex commands into functions.)

If we want to ignore Advice #2, the resulting Dockerfile snippet would be

RUN apt-get install -y gcc\
 && gcc --version\
 && apt-get --purge autoremove -y gcc

which is not so easy to read and maintain because of the obfuscation. See how the shell-script variant outs emphasis on the important part gcc --version while the chained-&& variant buries that part in the middle of noise.


So it turned out that some of the workers didn't have the correct credentials to the docker registry. After I applied the correct ones, the command worked normally.

That's weird, since I could do it locally...
Anyway, case solved.


What are some effective questions to ask someone who is considering me as their DevOps person?

In my opinion since the DevOps culture is derived from, or you could say appeared because of, the Agile software development methods

I would ask if the company is following any agile software methods or else you would be just the release guy
Also I would ask how I will be working as part of the dev team like I will be assigned to one project team or I will be working with all the teams
Also it would be great to know the current situation like they are currently facing urgent problem and need fix now , what's their current stack , and also ask about their current workflows in git and in development and try to find a way to solve the issue to make everything go smoothly
Also what really appears on paper is the workflows like git workflows, how often you release cause automating parts like delivery, and integrations,......etc really means less time and better quality and that will appears on paper even for the business guys

But at the end if the day every company is different than any other and every situation is different than the other so it's all back to your judgement to be comfortable in your daily job.


How to connect to postgres container from a node container using docker-compose?

docker-compose contains an internal DNS. In the example the service names are also the DNS names. In this case, the DNS of the database is db. If one configure this as a hostname in the node app then one should be able to access the database.

Back in the day, the wordpress example of docker-compose helped me understanding this. When I changed WORDPRESS_DB_HOST: db:3306 to WORDPRESS_DB_HOST: db2:3306, the wordpress was not able to access the mysql db anymore as depicted in the log. When it was changed back and forth to db and to db2 it was not able to access it and with db it was. Then I realized that the service names are actually added to an internal DNS that can be resolved by other services that are defined in the docker-compose.yml.

Recently I realized that running docker-compose up and specifying multiple files using -f also ensures that all services get added to the internal DNS. Initially I thought that the resolving was restricted to the same docker-compose.yml, but probably the docker network is responsible. If multiple apps reside in the same network, in this case the default, then they can resolve each other.


How to get environment variables in ECS container?

Environment variables are defined inside the container and some are passed in the task definition. So you can use describe-task-definition to see the extra variables.

To get the default environment of the container image, you can register-task-definition for task with command 'env' and the specific container image for which you want to find out and then run-task this task.


Why does chown not work in RUN command in Docker?

I found the reason of the problem.
In Docker "anything after the VOLUME instruction in a Dockerfile will not be able to make changes to that volume".
I define volume "./app:/var/www/app", and after that manipulate with it, so it's not work.


How to fix a "heartbeat failure" in Docker Swarm?

This might not be the answer to your specific cross data-center-ip setup.

I occasionally run into one or more swarm-nodes being status: Down and Availability: Active. Having the  Status.Message: "heartbeat failure". This can happen after a reboot.

What helped was to stop the docker daemon, remove /var/lib/docker/swarm/worker/tasks.db and start the docker daemon again.

from:
https://github.com/moby/moby/issues/34827#issuecomment-457678500

sometimes it fixes itself: https://stackoverflow.com/a/54126180/2087704


How to performance test AWS Lambda functions?

Dawny33 response is good, but I would start earlier in the development process.

Keeping an eye on your cloud environment to make sure that your functions behaves as you expect (including your "production" functions, which might operate on a different data set) is crucial, because it might reveal things that are impossible to reproduce locally or with a test data-set.

Nonetheless, I would say that this performance testing that you do in an optimization purpose should start straight from the developer's machine. Or, at least, from some local environment before pushing to the cloud.

The reason I say so is that while AWS Lambdas are amazing on many points, the fact that you don't have a full control on the server will limit your instrumentation abilities. I'm not saying instrumentation is impossible when in serverless, but try figuring out how many CPU interrupts you have (and how many are caused by your code) just for fun ;)

So what I advise, and this is actually not limited to serverless, is to start the profiling early. NodeJS profiling can be made with many different tools, NewRelic, dynatrace and AppDynamic are some of the big players. There is also smaller player, some of them are just a NPM package to install (like Nodefly). It is also possible to do some NodeJS without any additional tool, as there is a profiler built into the V8 engine. This documentation from NodeJS will get you started.

Whatever tool you choose, you want to install it locally and collect profiling data. That might involve running an agent or including a package in your package.json. Your tool's instructions will tell you how to install it. A good profiler will let you know how much memory and CPU you use. Better tools will give you insight on how many remote calls were made, how long they took.

Use the profiling data that the tool gives you to identify bottlenecks and address them. There's no limit on how much profiling you can make. Some people (crazy?) will look at the system calls of their most critical function. You might have to do that kind of thing if you want to shave nanoseconds of your function (but then, maybe AWS Lambda isn't the best choice to start with).

It's also worth noting at this point that I haven't mentioned anything specific to AWS Lambda. That's because your optimizations will most likely not be AWS Lambda specific (after all, in serverless you shouldn't be worried about the server/environment).

Make sure that not only your code works, but that it works the way you expect it to. Don't over optimize, but keep a critical eye on CPU and memory usage. Should a 2MB array really grow to 10MB when you sort it? Probably not.

Then you will be able to use tools mentioned by Dawny33, or some other tools, to confirm that your functions perform similarly when deployed to Lambda. You will however already have a very high level of confidence in your function and will only need to validate that they behave properly, not fully profile them.


How do you limit access to the production environment in the company you work for?

If your question is how to manage database changes consider something like Flyway. This lets you control your changes via tracked config files in your repository and apply them via an automated & controlled process - use your normal code review & promotion steps.

If the question is "how do I get devs to stop bugging me to run arbitrary SQL commands" then you may want to consider scripting out a way to automate it or giving them a 3rd party UI to use with an account that's locked down to prevent changes & restricts them from seeing any sensitive tables. YMMV depending on the layout of your DB.


The problem is twofold:

The Win10 LTSB is on its newest version and Docker has stopped supporting this version.
This stops Docker from updating.
Microsoft has moved to its own container registry and in doing so abandoned support for older Docker versions. So while I was pulling from the right registry it could not find a fitting image.

Solution: get an up-to-date, non-LTSB version of Windows

Additional Info:
While the error message did not help in finding the cause the docker log did:

[WindowsDaemon  ][Info   ] debug: a Windows version 10.0.18363-based image is incompatible with a 10.0.14393 host


Software/devops engineer's personal dashboard recomendations

For all of your read-only dashboard needs, there is Grafana with multitude of plugins, data sources and visualizations (tables, charts etc). You can also program your custom data sources pretty easily with official "simple-json-datasource" plugin.

By default, Grafana does not support buttons or other kind of form/action stuff on its own. Grafana is all about visualizing data, not modifying it. Yet, while I haven't tried it myself, Grafana supports Text/HTML/Markdown panels, I think you can implement any JavaScript logic inside those panels.


I ended up using stolon for its automatic leader election and fail over handling.
Crunchy Data offers small building blocks so there is a lot of work to do to put them all together while stolon has a ready helm chart.
But I didn't try patroni since stolon worked just fine.


Note: I didn't use this yet, the answer is based solely on documentation.

A GitlabCI pipeline can be triggered via API, see Triggering pipelines through the API.

  Triggers can be used to force a pipeline rerun of a specific ref (branch or tag) with an API call.

You would first need to create a trigger for that pipeline.

Then you can activate that trigger (from your jenkins job in this case):

  To trigger a job you need to send a POST request to GitLab's API
  endpoint:

POST /projects/:id/trigger/pipeline

  The required parameters are the trigger's token and the Git
  ref on which the trigger will be performed. Valid refs are the
  branch and the tag.

Such triggers support variable passing:

  You can pass any number of arbitrary variables in the trigger API call
  and they will be available in GitLab CI so that they can be used in
  your .gitlab-ci.yml file. The parameter is of the form:

variables[key]=value

You can use the variables inside .gitlab-ci.yml like this:

upload_package:
  stage: package
  script:
  - if [ -n "${UPLOAD_TO_S3}" ]; then make upload; fi

And

  You can then trigger a rebuild while you pass the UPLOAD_TO_S3
  variable and the script of the upload_package job will run:

curl --request POST \
  --form token=TOKEN \
  --form ref=master \
  --form "variables[UPLOAD_TO_S3]=true" \
  https://gitlab.example.com/api/v4/projects/9/trigger/pipeline

This kind of POST requests is what you would to do in your jenkins job. But take a closer look at the triggers, you might find a way to do what you want without jenkins.


Create multiple EC2 instances using python script (in loop iteration)

The main issue I see is that you are calling Terraform at the end of every python loop. This is creating new "state" file. That will make it really hard to manage what you are creating. This is how I would do it.

users = []
count = 0
for customer in data['customers']:
    username = customer['email'].split('@')[0]
    instances = customer['instances']
    count += instances
    users.extend([username] * instances)
    #print(Added {} to build list with {} instances.".format(username, instances))
myfunc(count, users)

then I would change...

def myfunc(count, users):

  tf = Terraform(working_dir='/home/ja/terraform-course/demo-2b', variables={'count':count,'INSTANCE_USERS':users})
  tf.plan(no_color=IsFlagged, refresh=False, capture_output=False)
  approve = {"auto-approve": True}
  print(tf.plan())
  print(tf.apply(**approve))
  return

and finally the Terraform.

resource "aws_key_pair" "mykey" {
  key_name = "mykey"
  public_key = "${file("${var.PATH_TO_PUBLIC_KEY}")}"
}

resource "aws_instance" "win-example" {
  ami = "${lookup(var.WIN_AMIS, var.AWS_REGION)}"
  instance_type = "t2.medium"
  count="${var.count}"

  key_name = "${aws_key_pair.mykey.key_name}"
  user_data = <<EOF
<powershell>
net user ${var.INSTANCE_USERS[count.index]} '${var.INSTANCE_PASSWORD}' /add /y
net localgroup administrators ${var.INSTANCE_USERS[count.index]} /add

winrm quickconfig -q
winrm set winrm/config/winrs '@{MaxMemoryPerShellMB="300"}'
winrm set winrm/config '@{MaxTimeoutms="1800000"}'
winrm set winrm/config/service '@{AllowUnencrypted="true"}'
winrm set winrm/config/service/auth '@{Basic="true"}'

netsh advfirewall firewall add rule name="WinRM 5985" protocol=TCP dir=in localport=5985 action=allow
netsh advfirewall firewall add rule name="WinRM 5986" protocol=TCP dir=in localport=5986 action=allow

net stop winrm
sc.exe config winrm start=auto
net start winrm
</powershell>
EOF

  provisioner "file" {
    source = "test.txt"
    destination = "C:/test.txt"
  }
  connection {
    type = "winrm"
    timeout = "10m"
    user = "${var.INSTANCE_USERNAME[count.index]}"
    password = "${var.INSTANCE_PASSWORD}"
  }

tags {
Name="${var.INSTANCE_USERS[count.index]}"
}
}

What I did was change your code so we now create a list of the users, that match the length of the count. That way we can call Terraform just once, and use the count.index on our users list to make sure we create the correct amount of instances per user.

You can read more about count.index here. I have always found reading and understanding more complicated code makes me better. Try reading some of the modules from the Terraform Registry. The vpc module gives good examples of using count.

If you still need help drop me a comment.


simulate SQL DB usage pattern of a typical cloud native web app with sysbench?

... [I] can't say what exact app (shop, historic info, social media, forum)

This makes this question very difficult to answer. This is because you have two basic types of queries:

OLTP and OLAP queries

  Online transaction processing (OLTP) is information systems that facilitate and manage transaction-oriented applications, typically for data entry and retrieval transaction processing.1

These kinds of queries tend to be very high volume, but short, and targeted and are a great fit for a system like MariaDB. On the other hand,

  Online analytical processing, or OLAP, is an approach to answering multi-dimensional analytical (MDA) queries swiftly in computing. OLAP is part of the broader category of business intelligence, which also encompasses relational database, report writing and data mining. Typical applications of OLAP include business reporting for sales, marketing, management reporting, business process management (BPM), budgeting and forecasting, financial reporting and similar areas, with new applications coming up, such as agriculture.2

These types of queries perform extremely poorly at scale in Maria DB unless it is properly tuned and certain database management and retrieval techniques are used (such as partitioning, for example). These performance issues are largely a result of a confluence of disk I/O issues and the fact that queries are single-threaded in terms of CPU. While sysbench can do a great job of testing the performance of OLTP queries, for OLAP queries, you will need something else. For example, one programmer forked sysbench to add the ability to do OLAP queries.

Conclusion

Unless we can get a bit more information about how much data you are working with and what kind of I/O operations you will be doing (reporting or transactional) it is hard to say if this specific tool will do the job. Ultimately database management is a complex discipline and there is a reason DBAs are able to make careers out of just database management. I have seen a lot of programmers think that they don't really need to worry about database management and they don't really need to integrate a DBA into their team. As their app scaled, they very quickly found out they were dead wrong. If you are planning a multi-tenancy DBaaS environment, you are going to need to architect your system very carefully in order to prevent one tenant from affecting the performance of the other and consider offering DBA consulting if your customers begin to suffer performance issues - and factor that into the price. If they can't afford their own databases and find that managing that in house is untenable and unaffordable, you will need to provide both the platform and the expertise as part of your service as one customers performance issue can easily become all of your customers' performance issue. That means you are going to really need to know your stuff or make sure you hire people who do. In short, databases are hard.

1 https://en.wikipedia.org/wiki/Online_transaction_processing

2 https://en.wikipedia.org/wiki/Online_analytical_processing


Why am I unable to push docker image in gce (gcr.io)?

From the Before you begin section of the instructions you referenced:

  Make sure that you:

  Have access to the registries which you will be pushing to and pulling from

This is specified in more details in Using Container Registry with Google Cloud Platform:

  To push private Docker images from a Compute Engine instance, your
  instance must have read-write or full-control permission to
  the image's bucket.

So the GCS bucket corresponding to gcr.io (or whichever GCR domain you want to use) and the desired cloud project must already be created and your GCE instance's service account must have the necessary role/permissions for push operations.

It's true, the instructions you referenced mention in Push the tagged image to Container Registry:

  When you push an image to a registry with a new hostname, Container
  Registry creates a storage bucket in the specified multi-regional
  location.

So I believe you'd have to:

first create create the bucket by executing a push to the desired GCR domain and project, but:

from your own computer (or some machine which is not a GCE instance, but YMMV)
using the credentials of an acceptable identity, not the GCE instance service account's ones. I'd try with a real user credentials, I'm not sure if a service account's ones would cut it.

after the bucket is created add to its access control the GCE instance service account's role/permissions for push operations
then perform pushes from the GCE instance


Tools used in enterprise level to transfer build artifacts from CI server to live server

Try LFTP that has commands to mirror or parallel copy. LFTP can run over many protocols.

The likely problem you have is that you are transfering a lot of data from a build service to a single vm over the Internet. Enterprises use the same tools and protocols but they pay for more bandwidth else ensure that their builds and deploy happens within the same cloud provider. They then deploy to many servers in parallel using standard protocols like SCP with tools like Ansible. The networks within a single cloud provider (Azure, AWS, GCP) are prioritised. This means of you want fat deployment bandwidth, and to run on Digital Ocean, you should build on Digital Ocean.

It may be the case that a very large amount of the 50k of files dont change on each build. So what you really want is a tool that looks at what files are already deployed. We can expect such tools to use file name, timestamps, file sizes and checksums to check whether a file needs uploading. If your build is stateless and creates 1G of fresh files each time their timestamps are going to be newer that the identical files already deployed. LFTP has an "ignore time stamp" option to avoid transfers of newer but identical files. See this question and answer https://serverfault.com/a/311670


Should I run CI on merge state or HEAD of branch in a PR?

Usually you want to test your code as closely as possible to how it's going to be deployed.  This means you're going to want to run your tests against code that's been merged with master, so choose the first option (merge first, then test).  There's no point in the 2nd option if you're never going to deploy the code in that state (i.e. unmerged).

Or for another perspective: good CI practices involve merging soon and merging often.  This would naturally lead one to the first option.


Constructing Dependencies from a Pillar in Saltstack

This templated SLS file works splendidly:

{% if 'components' in salt.pillar.items() %}

include:
{% for component in salt.pillar.get('components').keys() %}
  - {{ component }}
{% endfor %}

{% endif %}

However, it requires changing my pillar key structure. From the original question, you see the structure as:

my-minion-id:
    ----------
    components:
        - a-dependency-name

Instead, it needs to be:

my-minion-id:
    ----------
    components:
        ----------
        a-dependency-name:
            None

Note that a-dependency-name is now a dict, and has a single key/value, None. Thus the pillar SLS file needs to change from

components:
  - a-dependency-name

to

components:
  a-dependency-name: ~

While you can get away with not using no-value dictionaries (and thus get rid of the .keys() in the template) and use lists instead, if you do that, you can't merge the components from multiple different pillars; each pillar Salt applies will override the components key from the previous, and the last pillar read will win. If you want to merge pillars (we are talking about role-based assignment, here), this would appear to be the necessary construct.

With that, the output of salt my-minion-id state.show_sls components is then correct:

$ salt my-minion-id state.show_sls components
my-minion-id:
    ----------
    a-dependency-name:
      ----------
      ...

pillar.items() in Jinja templates turns out not to be quite equivalent to salt.pillar.items(); if you try using pillar.ls(), for example, you may see the following error:

Rendering SLS 'base:components' failed: Jinja variable 'salt.pillar object' has no attribute 'ls'

Whether that means one should avoid the implicit salt. prefix shortcut Jinja provides, or instead use a construct like {% if pillar['components'] is defined %} (thanks, @brousch, for the advice), I can't say.


how to set a variable from an ini file in ansible?

Since the ini lookup plugin only reads files local to the controlling host, the remote file needs to be fetched locally before it can be read and set_fact has to be used to put its content into a variable, which results in the following:
- block:
  - name: "Retrieve remote ini file"
    fetch:
      src: /path_to/ini_file
      dest: /tmp/ansible
  - name: "Read and store my value"
    set_fact:
      my_variable: "{{ lookup( 'ini', 'myParam section=mySection file=/tmp/ansible/{{ inventory_hostname }}/path_to/ini_file' ) }}"

Then the parameter is available as variable and can be used further in the playbook
- debug:
    var: my_variable


In short, Chef may not be the tool to use here.

Chef is a convergent model, so you have to write your recipes in an idempotent way such as after some runs it will be at the desired state according to the others nodes around.

Your approach about storing the state sounds the way to go, and you'll have to use an external orchestrator to schedule the chef-client runs alternatively on A and B.
If you have a chef-server, push jobs could be the way to go sticking in chef ecosystem, main advantage I see with it compared to others orchestrators is a pull model not needing an incoming administrative port nor ssh on the target nodes.


In my opinion, there should always one key per user:

Security: If an employee will leave the company then the key could be revoked and the person who left the company cannot trigger builds anymore
Access-control and responsibilties: developers from team b are allowed to deploy apps to the production environment of team b, while members of team a are responsible for their own app and are restricted to environment a.


I suspect that Bitbucket pipelines is enforcing this memory limit, not docker-compose.  This is nothing to do with the Linux kernel's OOM killer.

The page you linked is about memory limits in additional containers (e.g. for a database) used by the pipeline, not the pipeline itself.

Some suggestions:

Try editing bitbucket-pipelines.yml to include a size option for memory size - I would expect this is more useful than editing Docker-compose options which may not be respected.  See this doc page.
Try running the build locally under Docker Compose to get an idea of memory usage.

It would help to include the details of your bitbucket-pipelines.yml, and the Bitbucket error message about being out of memory.


One time database update process as part of applying a patch

This is usually called a migration, try yii2, Laravel or doctrine

Mostly these frameworks will manage the schema in yml, XML or PHP and automatically calculate schema changes and apply them.

For data changes, you would package up a number of database queries or even program code into a migration, which usually has a forward/up step and a backward/down step.

Migration status is usually stored as a single value in a separate database table


Ansible playbook to ensure Ruby gems are installed


How to use stored certificate in Jenkins declarative pipeline?

I usually set the credential NAME into an environment variable

environment {
    CERT = 'mycert'
}

Then I can use that name where ever I need it

withCredentials([usernamePassword(credentialsId: "${env.CERT}",
                                  passwordVariable: 'CERT_PASSWORD',
                                  usernameVariable: 'CERT_USER')]) {
    // Do things needing environment variables
}

or

checkout(poll: false,
         scm: [
               $class: 'GitSCM',
               userRemoteConfigs: [[credentialsId: "${env.CERT}",
                                   url: 'https://code.example.org/somerepo']]
               ])


How do you hide sensitive information held in variables on a docker-compose file

Use environment variables on your host that are read by your docker-compose.yml file.  There are a couple ways to accomplish this.  One way is to create a .env file on the host that contains your variables.  Docker Compose will read these variables and apply them to your configuration.

Alternatively, or in conjunction with a .env file, you can set environment variables in your shell.  These variables will be applied the same way as those read from a .env file and actually take precedence.

Read more here: https://docs.docker.com/compose/environment-variables/


Azure Windows Container: How to read secrets from an azure vault

Assuming this is transient container then my suggestion to work around the not yet implemented issues would be that whatever creates it does so by passing in secure environment variables for the values you want from the key vault; then your scripts can access the data that way.

I'm not sure what your security considerations are so you'll need to take that into account so people can't login to the container to view the environment variables.

As an example I have a function that uses the .Net Fluent API to create a container group and then sets the environment variables for the container using ".WithEnvironmentVariablesWithSecuredValue()" this means anyone whose looking at the container in the portal won't be able to see the values in plain text.

Then you assign a managed identity to the function and then give that identity get permissions to secrets in the key vault and then you can either access the key vault in code in the function or the easier option is the put them as app settings for your function and then just get them as environment variables in your function - https://docs.microsoft.com/en-us/azure/app-service/app-service-key-vault-references

Hope this helps or gives you an idea on a workaround?


Interference of Docker CMD with su: works with su-exec but not with su?

wrong argument order to su

$ZOO_USER needs to be the first argument to su with the "$@" coming afterwards.

su -c "$0" "$ZOO_USER" "$@"

It is ok for the options to come before the username.  It is a good idea to keep them in double quotes so that there aren't issues if someone accidentally puts spaces into it.

Your error message reinforces that this is an order of arguments problem.

even better

Based on comments this would be even better:

su -c "$0" "$ZOO_USER" -- "$@"

su man page excerpt

  Additional arguments may be provided after the username, in which case they are supplied to the user's login shell. In particular, an argument of
  -c will cause the next argument to be treated as a command by most command interpreters. The command will be executed by the shell specified in /etc/passwd for the target user.

  You can use the -- argument to separate su options from the arguments supplied to the shell.

busybox docs excerpt:

In a comment the OP asked about why this wouldn't work in Alpine Linux.  Since alpine is based on busybox I looked up the busybox docs.
 According to the docs busybox doesn't support sending additional arguments to su:

su
su [OPTIONS] [-] [username]

Change user id or become root

Options:

        -p, -m  Preserve environment
        -c CMD  Command to pass to 'sh -c'
        -s SH   Shell to use instead of default shell


Jenkins Pipeline job; proper quoting for slackSend step

In Groovy you have to use double quotes to get string interpolation:

    if ( "$output" != null ) {
        slackSend (channel: "@${name}", color: '#36A64F', message: "Job succeeded")
    } else {
        slackSend (channel: "@${name}", color: '#36A64F', message: "Job failed")
    }

You can also probably do something like this to avoid it entirely and have less duplication:

name = '@' + readFile('name.txt')

slackSend (channel: name //etc.


Terraform and Kubernetes' StorageClass with node affinity

Well of course its always the same - as soon as you actually do ask for help you figure it out...

resource "kubernetes_persistent_volume" "nodeconfig" {
  metadata {
    name = "nodeconfig"
    labels = {
      a = "a"
    }
  }
  spec {
    capacity = {
      storage = "25Gi"
    }
    storage_class_name               = "nodeconfig"
    persistent_volume_reclaim_policy = "Retain"
    access_modes                     = ["ReadOnlyMany"]
    persistent_volume_source {
      local {
        path = "/mnt/nodeconfig"
      }
    }
    node_affinity {
      required {
        node_selector_term {
          match_expressions {
            key = "kubernetes.io/hostname"
            operator = "In"
            values = [var.config.RGs.Primary.clustervmhost]  # must mnanually put this one in
          }
        }
      }
    }
  }
}

So the docs were a bit off of the lists of lists, it was a map of maps of lists.


It’s possible to use Publish Over SSH Plugin. Lets set up ssh connection details in the settings interface of Jenkins. We assign server a name, an address, user’s name and authorization way — by password, passphrase orindicatedcate key. These adjustments are simple and easy. You may found more details the on plugin’s page.

After Install Plugin Add remote server

After Add remote server you can find in Build section Send file or Execute command over SSH

Here you can write command for Tomcat start/stop/restart

for more details visit this URL: https://medium.com/@weblab_tech/how-to-publish-artifacts-in-jenkins-f021b17fde71


Jenkins input pipeline step - Give submitter as a group

We have moved onto using Jenkins templates and I have a workaround to keep the individual approver names list in one place. Sharing the sample code for your reference.

#!/usr/bin/env groovy
import com.mm.Constants
def call(String teamName = 'LOGIN') {
   // team name of null means UI
   teamName = teamName ?: 'UI'

   timeout(time:2, unit:'DAYS')
       {
           input message: 'Can you approve this ??', submitter: Constants."${teamName}_APPROVERS"
      }

}

#!/usr/bin/env groovy
class Constants {
   static final LOGIN_APPROVERS = 'approver1,approver2';
   static final UI_APPROVERS = 'approver2,approver3';
}

Below is how you call from the pipeline

stage ('Ship to QA?'){
    echo "Waiting for QA approval"
    shipToQA('LOGIN')
}


Is it possible to set a minimum Python version in Pygradle?

Side note: personally I dislike using a minimum version: you never know beforehand how the language will evolve and if a certain future version will still be compatible with your code. For example many 2.7-compatible scripts can't actually work with python 3.X, setting 2.7 as the minimum version won't cut it.

I prefer to explicitly list as allowed the (existing) versions that I can test and confirm to be compatible. Any future version can be added later on, after such confirmation is obtained.

If you agree with the above then the com.linkedin.python plugin can apparently perform such version checks (it doesn't looks like it supports a minimum version):

  Default and allowed Python version

  This plugin enforces a set of default and allowed Python versions. For
  example, you can specify pythonVersion = '3' and you will get
  whatever the default Python 3 version is. Similarly pythonVersion =
  '2' gets you whatever the default Python 2 version is.

  This plugin also enforces a set of allowed Python versions. If you
  choose a Python version that is not allowed, you will see an error
  messages such as:

Python 3.2 not allowed; choose from [2.6, 2.7, 3.4, 3.5, 3.6]

  If you see this error message, you must adjust your pythonVersion
  setting to one of the allowed values.

  Consumers of this extension can change the default Python 2, Python 3,
  and allowed versions by making a call on the PythonDetails object,
  either the one returned by PythonExtension.getDetails() or on any
  PythonDetails instance you create, e.g. in Groovy:

pythonDetails.setPythonDefaultVersions('2.7', '3.6', ['2.7', '3.5', '3.6'])

But I'm not familiar with pygradle, I can't really tell exactly how the plugin works and/or is configured to do the job.


How can I add an additional IP / hostname to my Kubernetes certificate?

To do this, you’ll first need your kubeadm configuration file. This creates a file named kubeadm.yaml:

kubectl -n kube-system get configmap kubeadm-config -o jsonpath='{.data.ClusterConfiguration}' > kubeadm.yaml

Now open the file in an editor, and find the certSANs list under the apiServer section. If it does not exist, you’ll need to add it; if so, you’ll just add another entry to that list. Example:

apiServer:
  certSANs:
  - "172.29.50.162"
  - "k8s.domain.com"
  - "other-k8s.domain.net"
  extraArgs:
    authorization-mode: Node,RBAC
  timeoutForControlPlane: 4m0s

Now move the old certificates to another folder, otherwise kubeadm will not recreate new ones:

mv /etc/kubernetes/pki/apiserver.{crt,key} ~

Use kubeadm to generate new apiserver certificates:

kubeadm init phase certs apiserver --config kubeadm.yaml

Now restart your kubeapiserver container:

Run docker ps | grep kube-apiserver | grep -v pause to get the
container ID for the container running the Kubernetes API server
Run docker kill <containerID> to kill the container.
The Kubelet will automatically restart the container, which will pick up the new certificate.

If everything is working as expected, don't forget to update the kubeadm ConfigMap stored in the cluster, otherwise, future kubeadm upgrade will be lacking your new config:

If using Kubernetes < v1.15:

kubeadm config upload from-file --config kubeadm.yaml

For Kubernetes version >= v1.15:

kubeadm init phase upload-config kubeadm --config kubeadm.yaml

This article has a more complete guide on how to Adding a Name to the Kubernetes API Server Certificate


How to set Deployment Type using ECS First Run Wizard

I think you want to setup Blue/Green deployment type for ECS. As Blue/Green deployment, there should be one environment (Blue) should exist before deploying another environment called Green. As it's your first project or first ECS services, there is no prior environment thus we cannot choose Blue/Green deployment in deployment type.


How to create a preemptable docker container?

I'm not aware of any straightforward way to do that, without also running a different container to act as a middleware of sorts. You might want to look at one of the "Serverless" platforms. If your use case will fit that model, many of the docker/swarm/kubernetes serverless tools do what you're asking, but again, you're running other container(s) to manage it. I'd suggest starting with a glance at the OpenFAAS docs to see if it'll work for you.


PHP-FPM with Nginx in different containers (share init.d response between)

Here is nginx and php done as an s2i builder image. You can basically have plain old php in a git repo and run s2i to make a runtime image that runs it with nginx and php. You simply docker build that dockerfile and pass the resultant image as the builder image to s2i along with your source code.

You will notice I have archived that project. That is because redhat have security patched image for php at https://access.redhat.com/containers/#/registry.access.redhat.com/rhscl/php-72-rhel7 they happen to use apache httpd but since I don’t see that I use tell s2i “make a runtime container for my php code in git” and it just works and I get frequent security patches it’s way easier than messing around with my own dockerfile to do security patches.


Does Docker allow containers to be OS agnostic?

Does the Docker engine abstract away the OS such that this configuration will run both apps?

No, it does not. Docker uses containerisation as a core technology, which relies on the concept of sharing a kernel between containers. If one Docker image relies on a Windows kernel and another relies on a Linux kernel, you cannot run those two images on the same OS.

When you install Linux Docker on Windows, you're actually setting up a Linux VM in Hyper-V, the LinuxKit details are here. You can open the Hyper-V console and see a VM called "MobyLinuxVM". There's a lot of work done to abstract this away from you e.g. the docker binary can connect to the Linux VM and run the same commands you're used to, networking between the host and VM tries to be seamless, volume mounting, etc. Ultimately it's not Docker that lets you run a Linux Docker image on Windows, it's just plain old virtualisation.


Is it possible to upgrade Jenkins from Jenkins it-self?

Yes, it is possible. In Windows, this is what I see when there is a new version available:

When you click the Upgrade Automatically button, Jenkins downloads and install the new war file itself.

Please, check Jenkins wiki for further details. There are some problems if you installed Jenkins from a Debian package.

  If you run Jenkins with java -jar jenkins.war, Jenkins will be able to update itself from the Manage Jenkins page. This is the simplest way to do automatic upgrades.

*nix/Mac

  If you've installed via the OS X installer, the jenkins running the process does not own the .war file. To get auto-upgrade working, just fix the permissions so the jenkins user can write the WAR (in /Applications/Jenkins).

Debian Package

  Using the Upgrade-Button from within Jenkins (available since 1.318) will not work if Jenkins was installed from a Debian package (results in permission denied errors when trying to download the new WAR file)!

Windows Auto-Upgrade

  If you install Jenkins as a Windows service, Jenkins will be able to update itself from the Manage Jenkins page.


Jenkins Groovy to parse console output and mark build failure

This question seems similar to another on StackOverflow, but seems to be a close, but not exact duplicate of
https://stackoverflow.com/questions/37018509/jenkinsfile-build-log

To quote those :

  if you just want to check, that your log contains string myTestString you can just call manager.logContains('.myTestString.')

  If you want to get some information from the first matching line you can use manager.getLogMatcher(regexp)

You could use the Groovy flow control to break the build - you just need a step to execute the test:

Following the example on the link :

node {
    stage('CheckLog') {
      steps {
        if (manager.logContains('.*myTestString.*')) {
          error("Build failed because of this and that..")
        }
      }
    }


Docker Deployment strategy for microservices project

I'll ignore your usage of the word "VM" here, since you have "Docker" in the title and "container" in the tags.

One would indeed not deploy each microservice in their own VM, which would be an incredible waste of resources.

But Docker is different. It is no VM. There is no virtualization. There is no extra layer between the processes running inside the Docker container and the host kernel. There is no significant overhead in the form of RAM usage. It is more or less the same as just running the process without a container.

Docker is based on the concept of namespaces. Modern Linux kernels have namespaces for all their internal data structures (from user IDs to network interfaces to process tables). All aspects regarding accessing kernel ressources/hardware/networking etc. are exactly the same, no matter whether a process runs inside a Docker container or on the host itself. That's the beauty of it, that's also why a docker container spins up very quickly with almost no overhead, and why the actual runtime also has no appreciable overhead.

There is one single difference, which is the filesystem; and this is not due to virtualization of any kind, but due to the "onion" style of layering many filesystems on top of each other. There are options when building images to avoid this, i.e. pack many layers into one, which would alleviate such problems (and conversely, you can mount unionfs-style filesystems in pure Linux as well, and those will have exactly the same performance overhead).

But incidentally his overhead is mostly negligible because something like your standard microservice will load once, and then just run. If you use storage in a container, you usually don't write it into the container itself (though you can), but in a separate volume mounted when spinning up the container; and those are like normal Linux mounts, i.e. they have no performance penalty.

TL;DR: Docker has in the most cases no relevant performance penalty at all; namespacing instead of virtualization is the reason why.


How to pass the version to docker-compose?

Another way to execute the @golfNintyNine answer is putting inside docker-compose.yml the environment variable:

serviceName:
image: "imageName:${VERSION}"

And then executing simply:

$ VERSION=2 docker-compose up

Also you can put all together inside script file like this:

#!/bin/bash

function usage {
    cat << EOF >&2
Version argument is required, please use -v or --version.
EOF
    exit 1
}

OPTS=`getopt -o v: -l version: -- "$@"`

if [ $? != 0 ] ; then usage ; fi

eval set -- "$OPTS"

while true; do
    case "$1" in
        -v | --version ) export VERSION="$2"; shift 2;;
        -- ) shift; break ;;
    esac
done

if [ -z $VERSION ]
then
    usage
fi

docker-compose up


TravisCI did not run after adding more configuration to .travis.yml file

One of the headaches of .yml files is the indentation. So the parse error I was getting in the Request tab was saying no newline added. I was not clear whether that meant it needed a new line or was missing a new line, but by running: wc -l .travis.yml I was able to detect that it was not outputting the correct number of lines.

I tried about five different ways to format the .yml file allowing it to keep me focused towards the end of the files line number where the error was pointing towards.

I happen to look up and realized the beginning of the deploy: script was not properly aligned, so I adjusted with a shift+tab and after I got it properly aligned, TravisCI recognized my changes and properly ran the script and successfully deployed the application to Elasticbeanstalk.


Gitlab CI/CD fails to build docker image

It looks like that the docker login and the docker build were done by different users. If a docker login is performed, then a ~/.docker directory will be created in the namespace of user X, if the build is done by user Y then the latter user does not have a .docker folder and is not authorized to pull the image. One could verify this by echo $USER in the pipeline, in this case gitlab-ci.yaml to check this.


I would recommend to create an issue about CD in the ticket system of salesforce. I have googled a little bit and this seems to be their issue tracker.

I would prevent to implement CD in salesforce as that will be very cumbersome and will take a lot of time. In my opinion this issue should be solved by SalesForce itself.


Where to install Prometheus for Kubernetes monitoring

This is more of an opinion based question but let me try to answer it.

installing Prometheus itself in the very same k8s cluster (as another running set of pods) is a normal thing.Am I right or just got a wrong idea from my research?

Considering good practices, shouldn't the monitoring tool be outside of the system that you actually intend to monitor?

This is a correct practice to have. Monitoring systems are usually setup to look into the object being monitored. (outside looking in). But with self healing systems like k8s, installing monitoring within the system can be done with minimum issues. There are a few cases you do need to look into for proper stability.

Pods get deleted :- Run Prometheus in a HA mode (2 replicas minimum)
so as not to lose any data.
Pods are replaced due to resource crunch:- Add priority classes
to prevent pod form being evicted.
Use Persistent volumes:- Look into volume node affinity
conflict. Also configure the data retention period to make
optimised use of storage.
Cluster goes down. See below.

Also, if I set up multiple Kubernetes clusters, isn't it more cumbersome to have one Prometheus installation per cluster?

Not at all, using a combination of IaC(Terraform/Chef),Config management(Ansible),CI/CD (Spinnaker),GitOps and Helm charts. Configuring/maintaining  Prometheus instances across multiple instances will not be an issue.

Is it possible to set up a single Prometheus installation and centralise all k8s clusters metrics on it?

Prometheus although highly optimised, will not be able to handle all the metrics thrown at it from multiple clusters.Also Prometheus does not scale horizontally on its own, therefore it would need huge resources. Centralising all metrics to one place will cause other issues as well.

Network Connectivity/Latency:- Running a centralised Prometheus
instance receiving data from over the public internet may cause data
loss/false alerts due to network drops and latency.
Cost:- Running tens of exporter all sending data over the internet
will cause high internet costs for cloud deployments.
Cluster managing:- You may face issues with exporter/job
configuration of such a large number of exporters.

Its a good choice to run Prometheus per cluster. wherein you can have all the data locally till the time is required.
Grafana does not consume much resources so its safe to run inside the cluster. However it faces a different set of issues:-

User/Permission Management :- Each local instance would need to be
maintained separately which is a huge task.
Charts Management:- Same as above.
L1/DevOps will have to keep n number of tabs open to monitor your
infra.

Other than this, setting up alerts on all the clusters separately would also be a never-ending task.
My advice would be to setup Prometheus clusters locally (inside the cluster or right next to it) and connect all of them via Thanos or Cortex
and set up a HA mode Grafana centrally on top of Thanos along with alerts and external storage.This will alert if the local Prometheus instance goes down and will help with the outside looking in factor

scenarios in which a longer history period might help. Any comments on this matter?

Depends upon the metrics being stored, A quarterly SLI/SLO and incident reports seem like a good use case to store data for longer than 15 days.


Snake bites its tail problem automating configuration of Certbot with Ansible

Don't read the text written on Certbot site, the real talk is in the documentation.
In order to make less actions, consider grouping configs by domains. Plays for a playbook in order to achieve what you want:
Deploy certbot and nginx with no config files, then for each domain on host:

Deploy an nginx config for default server with location /.well-known/acme-challenge { root /etc/letsencrypt/challenges; }
Reload nginx config
Check existence of a certificate for the domain in /etc/letsencrypt/live/domain
If cert is already presented, end the run
certbot certonly -n --web-root=/etc/letsencrypt/challenges --keep -d domain to make a cert request
Check for cert in a retry loop
Fix permissions for live certificates
Deploy normal nginx configs

You may deploy a stub config per domain instead of using the default server. The playbook could be run asynchronously for each domain.


How to split traffic in Google AppEngine for specific IP addresses

Yes, it is possible to split the incoming traffic between several GAE app/service versions based on the IP address from which the requests originate. From Splitting traffic across multiple versions:

  When you have specified two or more versions for splitting, you must
  choose whether to split traffic by using either an IP address or HTTP
  cookie. It's easier to set up an IP address split, but a cookie split
  is more precise. For more information, see IP address splitting
  and Cookie splitting.

Read the respective linked section (same page) for details and limitations.

Configuring the traffic split is done from the Versions page.


How to secure persistent user data with docker on client location?

I want to avoid situtations where someone with access to the machine can just copy out some user data, or even look at it.

You cannot stop someone with local access to the machine from reading data on the machine.  Full stop.


What's the impact of the General Data Protection Regulation (GDPR) on DevOps?

"Enforcing" means lawyers going after possible culprits. Frankly, in my opinion (and those of some folks around me which are more a mix between IT-guy and lawyer), nobody knows how that will work out exactly.

Those iLawyers expect a lot of very interesting court cases in the first months to tests out the water, and see what precedence cases are going to come up. There will probably be plenty of law firms fishing for nice fat fish to fry.

For me personally, I don't really see that much impact specifically regarding DevOps. The applications themselves need to be adapted as necessary, obviously. Also, the actual data storage, be it RDBMS, elastic search indexes or whatever, will have to be looked at.

Aside from that, one benefit of DevOps, specifically if you work with some kind of container system and infrastructure-as-code is that you should generally know exactly where your data, log files etc. is stored, and what kind of them you have; much more so than in classical servers, where your data and especially logs might be strewn all over the place. It should also be very easy to regularly roll your data as applicable, i.e., get lost of old stuff.

So it can help to, for example, take your catalogue of i-a-c files and walk through them diligently - you can then be pretty sure to have found everything you need to be aware of.


Should we build our application inside a docker container, or copy it in?

I understand this can be optimized to a level with multi-staged build.

This is an understatement - you do not "optimize" that, but that's just the way you do it.

To answer your question: yes, by all means run the build inside a docker container. How else would you be sure that at build time every library or dependency is the same? There is no reason whatsoever not to do it.

Of course you need to pay a bit of attention when writing the Dockerfile. Ideally, whatever you change in your source code day-to-day, the only thing that happens when you re-run your build is the very last step of actually compiling your code and doing whatever it is you do with it afterwards (creating your deliverable archive...).

You need to use multi-staged builds afterwards to get rid of all the intermediate steps, and especially all the compile-time-only components (and it's pretty easy as well).

Note that there's usually some parts of your setup which are used by both the compile-time and run-time (unless you are just creating a .war file and sending that off to a webserver). I.e., if you have a ruby-on-rails application, the "ruby-on-rails" part could well be shared between compile-time and run-time. You want to make sure to put everything that is shared in this way very early in the process, and have a clean cut-off point that you can use as a FROM later on, copying the compiled files in as needed.


what you are looking for is compliance and security for containers.  There are companies that can do this out the box for you, and even though I'm not advocating for either you can see what they offer and use that as a template.

https://www.twistlock.com/platform/container-compliance/
https://www.aquasec.com/use-cases/container-auditing-compliance/

As a HIPAA security officer and DevOps manager, I've been reviewing these processes.  The way to think of DevOps in your sense is probably the automation and culture of ship fast, fail early and fail often.  The goal is to not disrupt the pipeline from security and compliance.  Technically this looks like adding automated compliance checks in the pipeline before, during and after go live.  As automation gets built, you can replace the automation with a manual process'.  You can check out subjects like DevSecOps which is the same concept of ensuring compliance and probably closer to what you'll need to do.

here's an AWS diagram that might help https://aws.amazon.com/blogs/devops/implementing-devsecops-using-aws-codepipeline/


Jenkins pushing to branch overwrites commits made in interim

It really depends on your situation, but I'll go with the generic/naive approach and say do exactly what your developers do in the same situation: I pulled an hour ago, I made some changes, now I want to push my changes, but other changes have been pushed in the meantime. I commit, pull/rebase, then push. Assuming that whatever Jenkins is committing isn't tied to the commit it built from, this is the same process it should follow to get the same result.


Your proposed snapshot-id is not syntactically valid.  When you tried hard-coding the value, you used a valid identifier.

Underscores are not allowed, and the identifier can't begin with a number.

Here are the constraints on snapshot identifiers, from the API Reference:

  DBClusterSnapshotIdentifier

  The identifier of the DB cluster snapshot. This parameter is stored as a lowercase string.

  Constraints:

  Must contain from 1 to 63 letters, numbers, or hyphens.
  First character must be a letter.
  Cannot end with a hyphen or contain two consecutive hyphens.

  https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_CreateDBClusterSnapshot.html


CircleCI manually approved Deployment with Rollback

The answer, no.

The best ways to rollback would be to use git revert, which then adds the rollback to history (which is a good thing), and then push that like any other commit. The other would be to use the git hash of the commit you'd like to rollback to and build it with the CircleCI API.

Right now, you could also attempt to rebuild and older build via the UI but this isn't suggest for deploys & Workflows. Too many variables for safe, reproducible deploys.

- Ricardo N Feliciano
Developer Evangelist, CircleCI


CloudFront -> ELB -> Pod running Gunicorn

Nginx will help you in performing the below items if you need some of them, Then I would say it make sense to add it to the stack

Optimize the caching (you can also cache dynamic content.
Restrict access to the services (based on ips).
Implement basic auth for some services.
Load balancing traffic between different posds or services.
Rewrite URLS.
Pass extra headers or configs to the backend services.


How to create credentials for Docker Hub?

You can generate an access token for docker hub by going here:

https://hub.docker.com/settings/security

And creating a new access token.

It's essentially the same process for organisational access tokens or for personal access tokens IIRC.

Once you have generated the access token you can literally just use that token instead of the password for most docker hub operations (ie. docker login --username $user --password $token)


How to customize a Jenkins job dsl method

What you want is a Jenkins shared library which will allow you to (among other things) define custom Pipeline DSL steps in a repository you own.  See the "Defining custom steps" section in that document.

Here's the example custom step from that document:

// vars/sayHello.groovy
def call(String name = 'human') {
    // Any valid steps can be called from this code, just like in other
    // Scripted Pipeline
    echo "Hello, ${name}."
}


No Checkbox to" Restart Jenkins when installation is complete".Unable to restart Jenkins post plugin download

To restart Jenkins manually, you can use either of the following commands (by entering their URL in a browser):

(jenkins_url)/safeRestart - Allows all running jobs to complete. New jobs will remain in the queue to run after the restart is complete.

(jenkins_url)/restart - Forces a restart without waiting for builds to complete.


With a bit trial and error I was able to work out the values for my "vnet_id" and "snet_id" variables.
I found using the full ID for the virtual network and not having a "/" before the subnet variable (snet_id) value fixed my problem.
At the moment I don't fully understand how Ansible/Azure puts it all together, but this has given me enough information to start to understand.
If anyone could point me to some documentation that explains the construction/interpolation of the variables it would be helpful.
  tasks:
   - set_fact:
       vnet_rg_id: "/subscriptions/REVMOVED-SUB-ID/resourceGroups/rg-ansible-test"
       vnet_id: "/subscriptions/REVMOVED-SUB-ID/resourceGroups/rg-ansible-test/providers/Microsoft.Network/virtualNetworks/vnet-ansible-test"
       snet_id: "snet-ansible-test"

   - name: Create resource group
     azure_rm_resourcegroup:
       name: "{{ rg }}" # declared earlier in the playbook
       location: "{{ location }}" # declared earlier in the playbook

   - name: Create virtual network inteface cards for VM
     azure_rm_networkinterface:
       resource_group: "{{ rg }}" # declared earlier in the playbook
       name: "{{ nic_name }}" # declared earlier in the playbook
       virtual_network: "{{ vnet_id }}"
       subnet: "{{ snet_id }}"
       location: "{{ location }}" # declared earlier in the playbook


How to avoid output being truncated when running AWS SSM command?

According to this ssm tutorial, SSM send-command output is truncated at 2500 characters.  To work around that, you must send your output to an S3 bucket using the --output-s3-bucket-name parameter.


Ideal methods of handling server logs on AWS

I like to use ELK, Elasticsearch, Logstash and Kibana with Beats.

Filebeat, that belongs to Beats series will forward the logs that read from a file to logstash.

In logstash you could tag, filter, parse and modify the log entries that are stored in Elasticsearch.

For visualizing you can use Kibana. You can create a great dashboards based on the logs with all the information that really gives value to your business

If you don't want to install agents, you can forward logs using rsyslog directly but with al beats series you could get more in deep information rather than simply logs.

This ELK suite could give you much control over your application but you must maintain it


Azure Devops - workflow/strategy for uploading only changed files to shared hosting provider after a build of static website (Jekyll)

I don't think you are missing any easy solutions because

the easy solutions probably rely on you having full access to the webserver -- which you don't have? The challenge is what is the best you can do given limited server access.
FTP does not have a reliable, built-in way to only upload changed files.

I think:

your only solution for problem 1 is to write your own shell script

problem 2 is usefully discussed here: https://stackoverflow.com/questions/424680/trying-to-find-a-simple-way-to-do-upload-only-modified-files-through-ftp .

The viable solutions both amount to 'save hashes of the files on the server, then compare before upload'. However if you can get your pipeline to install and use git-ftp then git-ftp will do all that for you.


A systematic way to test RESTful APIs with curl?

You could look at tools such as Postman which focuses on testing REST APIs with JavaScript - it has some nice features but you lose the use of Python.

Instead, I'd suggest looking at REST-related plugins for pytest, a Python test framework that simplifies your test code, while still running tests written using unittest.

For example, writing parameterised tests avoids boring and error-prone duplicate test code.

Pytest has a huge set of plugins that simplify various tasks, including:

Tavern, which specialises in testing REST APIs and seems highly relevant here - sort of "Postman for Python unit tests".
pytest-curl-report - when  testing with the requests library, will print a curl command you can use to reproduce the error from shell.
Testinfra - focuses on server testing (e.g. state of OS packages, files, processes, etc, usually tested on remote servers) - strongly recommended if you also need this type  e.g. to test Ansible code.

For those who use Puppet or Chef, Testinfra is similar to Beaker (with RSpec), ServerSpec or InSpec.

If you don't like Tavern, you can of course use pycurl with pytest, which makes it easier to diagnose exactly what failed. This example from the pytest-curl-report site uses only generic pytest features:

$ py.test test.py
============================= test session starts ==============================
platform darwin -- Python 2.7.9 -- py-1.4.27 -- pytest-2.6.4
plugins: curl-report, httpbin, cache, capturelog, cov, flakes, pep8
collected 1 items

test.py F

=================================== FAILURES ===================================
______________________________ test_requests_get _______________________________

    def test_requests_get():
        r = requests.get('http://httpbin.org/get')
>       assert False
E       assert False

test.py:7: AssertionError

Pytest lets you write all tests with a plain assert, and optionally include a helpful message as part of the output.  For example, one of your tests could be written:

def test_2():
    assert pycurl.response("some_url").matches ("xxx"), "xxx not found in response"


How to deploy Dev Azure SQL code to Prod Azure SQL using dacpac without knowing the target server credentials

You either have to have the login information in order to connect to the database, or, you have to configure your database server to allow access to Azure Services. This means that the SQL Database and server and the Azure DevOps pipelines have to be part of the same account (not necessarily the same login, but Azure AD). I'd lean towards requiring the connection information rather than open up everything broadly between all your Azure services.


What are best practices for using Ansible Vault on public CI's and Source Control Repositories like BitBucket?

--vault-password-file can instead point to an executable script that writes to stdout. This little-known feature should solve your problem.

First, write a simple, executable script that prints a system environment variable, and check that into your source control. Then, use Bitbucket's encrypted variable feature to set that environment variable to your ansible-vault secret. Finally, execute like so:

ansible-playbook site.yml --vault-password-file ./mypass.sh.

References:

http://docs.ansible.com/ansible/playbooks_vault.html#running-a-playbook-with-vault
https://groups.google.com/forum/#!topic/ansible-devel/1vFc3y6Ogto


Is there a Radiators for Jenkins and Bamboo?

It really depends on what information you are trying to "Radiate":

For an off-the-shelf solution that integrates with everything you mentioned you could check out Prometheus.  It has plugins for Jira, Bamboo, Jenkins, and SonarQube.

You could also opt for something quite a bit simpler and use the built in dashboards for the tools you are trying to display metrics for.  Just put them on different tabs and use an extension such as Revolver.

If you want to get a little more customized you can check out Elastic Stack's Kibana.  The book Agile Metrics in Action has several examples of how to capture and display Atlassian data inside of Kibana.


How to get logged in username of Jenkins in parameter's Groovy script?

From the Java API documentation for hudson.model.User:

  String  getDisplayName()
  Returns the user name.

  String  getFullName()
  Gets the human readable name of this user.

  String  getId()

It's not clear to me from your question what data you want (I think it's the user ID?), so I'm not sure which of these methods will get you that data (getId()?).  But in any case, you would call the method like:

User.current().getId()


Are you sure that your registry is publicly available?
Checklist:

Check if the domain is reachable and responding the right IP address -- nslookup myregistry
Check if your port is open or not blocked any kind of software (let's say via firewalls or AWS security group) -- telnet myregistry 5050
Check if your registry is not restricted for a particular CIDR of IP
Test your login locally and ensure that there is no problem with it


Ingress ip returns HTTP error DNS_PROBE_FINISHED_NXDOMAIN

There are two main reasons for this issue to happen:

Your dns are not exposed.
Your code is redirecting to a https address (and since you are working in your local machine, most likely you don't have a secure certificate)

CASE 1 - Exposing DNS
On linux you can solve it like:

minikube ip

vim /etc/hosts

Add the ip from the first step and the hostname you want to expose, and save the file.
# For example
192.168.39.70   elasticsearch-es-http

No need to restart any service, it should work just like that. Add all the hostnames you need.
CASE 2 - HTTPS on minikube
If your problem is 2, then you have to find a way for you application to not use HTTPS while you are using minikube.


The right way to Ansiblically deploy environments directly from Github:

There is no "right" or "wrong". If the commands work when you type them in, then they work, and we won't keep you from doing it.

Everything else is opinion. Sure, there are some best practices, for example some people find it unwise to directly fetch scripts from a public (3rd party) website and execute those locally without having a look inside first.

It goes without saying that your approach is in fact a rather significant risk. You are effectively giving root access on the target machine (and any other machine your local user has ssh keys for...) to anyone who can push into your repository. How earnest that risk is is something only you can decide.


The complication reside in the definition of the health status (and its usefulness).

You mentioned the definition being service is up and web app application is operational and for that indeed checking the homepage should suffice.

However that doesn't mean the service is properly configured for and correctly performing all the jobs that it's supposed to be doing. Without a consistent, universally acceptable definition of a health status there is no point of providing such information as a single, one-stop location to check.

But Jenkins offers via its Remote access API various checking points that users can combine to derive their interpretation of the overall service health status. For example maybe checking the status of all the configured jobs or at least that of the critical ones would be more useful, which could be done using the .../job_id/lastSuccessfulBuild/api/ or .../job_id/api/ checkpoints.

The JenkinsAPI could also be of interest (better docs at least), found via Where can I find jenkins restful api reference?


What does "eksctl create iamserviceaccount" do under the hood on an EKS cluster?

Yes It does more than just creating a service account. It does three things:

It Creates an IAM role.
It attaches the desired iam-policy (--attach-policy-arn
<POLICY_ARN>) to the created IAM role.
It creates a new kubernetes service account annotated with the arn of the created IAM role.

Now It's easy to declare the above steps using kubernetes and aws providers in terraform.


How do I add my IP address to the start of my terminal using the bashrc file?

Well... you would need to know what IP address you actually want. Is it a public IP address? Is it a private subnet address? Do you only have 1 IP address? A few things you would need to figure out.

But, if we just assume that we want to have whatever ip address your VM thinks it's address is then:

Add the following to your .bashrc

PS1="\[$(tput setaf 33)\][\u@$(ip -4 -o addr show scope global $(ip -4 route | grep default | head -n1 | awk '{print$5}') | awk '{gsub(/\/.*/, "",$4); print $4}' | head -n 1) \w ]\[$(tput sgr0)\] \[$(tput setaf 34)\]\\$\[$(tput sgr0)\] "

To throw an IP address into your prompt.

Now, because that's not really human friendly, let's break it down:

Make the prompt blue (honestly I can't remember why I always wrap with \[ \] I just remember someone telling me I needed to do that, so I guess see what works for you:

\[$(tput setaf 33)\]

This bit you already had figured out:

[\u@

This insanity to find the default IP address, it's a bit hard to break down without just reading through the ip man pages but basically we are searching for the interface marked as default then grabbing the IP address and stripping all the extra stuff out with awk/grep/head:

$(ip -4 -o addr show scope global $(ip -4 route | grep default | head -n1 | awk '{print$5}') | awk '{gsub(/\/.*/, "",$4); print $4}' | head -n 1)

I tend to want to know my ${PWD}:

\w ]

Reset our colour scheme just to be stafe (I've found weird stuff happens when you keep layering colours without resting:

\[$(tput sgr0)\]

Set some other colour:

\[$(tput setaf 34)\]

Set the $ or # prompt which is handy if you want it to be obvious if you are logged in as root:

\\$

Then reset colour again:

\[$(tput sgr0)\]


Would choosing the integration branch strategy be part of DevOps?

Branching strategy is a very clearly a business decision in which the DevOps engineer should be the primary expert, explaining all the trade offs. There are some strong positions on this issue in the community, very clearly Jez Humble is on one side with his book of Continuous Delivery. You could see him talking about it here. Here would be a video by panel of experts on the topic. And here is my highly up-voted answer on StackOverflow specifically on topic of feature branches. So clearly this is a big topic for DevOps.

One thing that people often say (Jez Humble mostly) is that feature branches are evil. You will need a (n-1)(n-2)/2 merges [combination number n-1 over 2] for n independent feature branches. Which comes to some very high numbers at 10+ feature branches. And because merging is hard and no publicly available version control systems do it will, then the entire Software Industry is moving away from software releases and software on premises to Software as a Service, where the only way is forward and software is both developed and maintained for you by the vendor.

As far as I know, there is only one company that can do high quality, business critical software on premises, with maintaining 10+ years of past major releases, back-porting patches through multiple waves of code refactoring and capable of working regularly on dozens of major feature branches per release and thousands small features developed on their individual feature branches and being continually merged as full features and not individual commits. The code has also been released on 14 different architectures. That company is Oracle, the product is RDBMS and they have been practicing what is now called DevOps since at least 1990. Hundreds of code deployments a day back in 2001 would be pretty standard. The reason why they are able to do it is through rigorous coding standards, invariant test suite and a proprietary version control system I've spent 10+ years of my life developing. A system that can do 99% of all merges through automation.

Everyone else pretty much only patches the most recent major release version past a certain stage/size of a project or it takes extreme amounts of human resources to merge all the patches to all the possible release versions and extreme amounts of human or compute resources to test them all. For example in Linux kernel you would have a maintainer for X.odd development version and later stability branch maintainer for X.even version, but you would see not patches for X-1.Y versions. Sometimes Linux distributions would maintain kernel version in their LTS release, but only at high cost and with varying code quality among distributions. And that is a high visibility project with enormous resources poured into it.

The primary reasons for this discrepancy are differences in:

Adherence to Coding Standards and Code Styles
Complete coverage by invariant test suite, which can catch bugs for which no tests were written specifically.
Version Control System capable of high levels of Automation.


How to solve `Error creating config for VM: Invalid JSON` in minishift?

One option to solve this issue is a suggestion as provided by @AndreasGajdosik, i.e. deleting the .minishift directory.

Once rm -rf ~/.minishift was issued and minishift status was run, the issue has been solved:

user@host$ minishift status
Does Not Exist


What tool is able to analyze images by connecting to clair?

Clair-scanner is able to scan the images locally.

https://github.com/arminc/clair-scanner/issues/13

docker run --net=host -d --name db arminc/clair-db:2017-09-18
docker run --net=host --add-host postgres:127.0.0.1 -d --name clair --net=host arminc/clair-local-scan:v2.0.1
./clair-scanner nginx:1.11.6-alpine example-nginx.yaml http://127.0.0.1:6060 127.0.0.1


Jenkins plugin site to ignore certificates

Following your comment confirming you can use your own signed certificates internally.

One solution would be to replace the current certificate on your repo with a new one holding updates.jenkins-ci.org as an alias. This response on stackoverflow has the necessary info to get you going.

Meanwhile, the plugin higlighted above by @casey vega in the comments (which I didn't know) seems to be a solution much easier to manage.


You're bringing up a big pain point - and there are 2 camps here, people who do GitFlow and people who prefer other approaches (with TBD being most popular).

Personally, TBD with release branches is something I would always prefer, but you can find lots of other people who would tell you GitFlow is better - so you'd have to decide for yourself.

In your case TBD would work as following - you keep committing to your master, however before release time you cut a release branch out of master. Once you do that, you don't have to code freeze master and your developers can keep working there.

Some materials for research:
On Gitflow - https://nvie.com/posts/a-successful-git-branching-model/

On TBD - https://trunkbaseddevelopment.com/ (especially check "Scaled Trunk-Based Development" section)

Critics of Gitflow (note that myself I'm biased towards TBD) -

https://georgestocker.com/2020/03/04/please-stop-recommending-git-flow/
https://medium.com/@mattia.battiston/why-i-love-trunk-based-development-641fcf0b94a0


How to test whether a variable exists in the declarative pipeline

use try/catch

def safeBuildUser = "unknown"
wrap([$class: 'BuildUser']) {
  try {
     safeBuildUser = BUILD_USER
  } catch (e) {
     echo "User not in scope, probably triggered from another job"
  }
}
echo "Builduser is: ${safeBuildUser}"


Why does my Docker composer not work with volume?

It seems there is a spacing issue in your yaml

You should change the compose file to

version : '3.4'

services:
    testando-volume-compartilhado-a:
      image: carloshenriquecarniatto/teste:latest

    volumes:
        - D:\App:/app  # <- extra space here
    ports:
        - "10001:80"
        - "44378:443"


How to copy files from a different linux server folder to Kubernetes pod

I have multiple files in a different linux server folder and I want to create a cron job and run daily in kubernetes that should copy the files to a pod network folder.

I am unaware how to perform the task as you have stated.

How to solve your problem? With the belief your "POD network folder" is a SMB, NFS, other type mounted resource;

I would implement one of the following, either will work the same in the end:

Establish a connection directly from the Linux Host (that presently has the file(s) to be copied) to the Networked Folder. Use SCP or Rsync to perform the copy.
Using SCP or Rsync copy the file(s) directly from the Linux Host homing the files to the IP Address of the POD. Given the POD has a standing connection to the Hosted Folder. e.g. NFS, SMB, other mount.


Gitlab CI/CD: deploy docker image to existing kubernetes cluster

Your error is most likely from referencing the wrong directory in your script block of your job.
When GitLab executes your job in a container, it mounts the project directory to the /builds directory in the container. You can reference this either by specifying /builds/scheduler-deployment.yaml or by using the predefined environment variable $CI_PROJECT_DIR with $CI_PROJECT_DIR/scheduler-deployment.yaml.


What is the best approach to deploying to multiple servers using Jenkins?

Have you looked at the Matrix project plugin?

Multi-Configuration Projects
A multi-configuration project is useful for instances where your
builds will make many similar build steps, and you would otherwise be
duplicating steps.
Configuration matrix
The Configuration Matrix allows you to specify what steps to
duplicate, and create a multiple-axis graph of the type of builds to
create.

Welcome to the Matrix is an excellent reference doc.
Example (from the blog) - Pipeline for multiple platforms and browsers

Jenkinsfile

pipeline {
    agent none
    stages {
        stage('BuildAndTest') {
            matrix {
                agent any
                axes {
                    axis {
                        name 'PLATFORM'
                        values 'linux', 'windows', 'mac'
                    }
                    axis {
                        name 'BROWSER'
                        values 'firefox', 'chrome', 'safari', 'edge'
                    }
                }
                stages {
                    stage('Build') {
                        steps {
                            echo "Do Build for ${PLATFORM} - ${BROWSER}"
                        }
                    }
                    stage('Test') {
                        steps {
                            echo "Do Test for ${PLATFORM} - ${BROWSER}"
                        }
                    }
                }
            }
        }
    }
}


docker "Cannot mkdir: Permission denied"

Try to run such command as root or grant user and group rights to the folder with chown.

That's only what comes to my mind for now.


gitlab -ci script not being updated on runner

OK, as usual the error was between the keyboard and the screen... I did use the "try again" button on the jobs list, which, of course was making use of the same commit, and therefore the changed made to the script were not taken into account ...


The layer that you're missing is nginx proxy. See my toy Mafia Game project here how to do it:

Shell deployment project (uses either Docker compose or k8s) - https://github.com/taleodor/mafia-deployment
Vue.js frontend - https://github.com/taleodor/mafia-vue
Express.js backend - https://github.com/taleodor/mafia-express
(Optional) detailed write-up about CI/CD and how it's built from DevOps prospective - https://itnext.io/building-kubernetes-cicd-pipeline-with-github-actions-argocd-and-reliza-hub-e7120b9be870

The missing piece in your case is nginx proxy that you use in your UI.
So what you need to do is to define some location for your api - i.e. /api. Then your proxy it via nginx - see my nginx configuration for the project above here - https://github.com/taleodor/mafia-vue/tree/master/nginx. At the same time it's highly recommended for performance to deploy static resources on the same nginx.
Use Dockerfile to define environment variables and defaults, i.e. in my project - https://github.com/taleodor/mafia-vue/blob/master/Dockerfile
Finally, you can update configuration via environment variables in your docker-compose yaml or k8s definitions.


How can I download an artifact from Jenkins using the Google Login Plugin?

Thank you to the comment from user54 for the answer.

Both of my pasted wget URLs work as long as I'm using the correct value for password. In this case, I needed to use my user's API Token as described in the answer to another question:
https://stackoverflow.com/a/34634247/317951


Changing the build intermediate paths for gitlab-runner

Conceptually, this approach is not the way to go; the build directory is not a deployment directory, it's a temporary directory, to build or to deploy from, whereas on a shell executor this could be fixed.

So what you need is to deploy from that directory with a script as per gitlab-ci.yml below, to the correct directory of deployment.

stages:
- deploy

variables:
  TARGET_DIR: /home/ab12/public_html/$CI_PROJECT_NAME

deploy:
  stage: deploy
  script:
     mkdir -pv $TARGET_DIR
     rsync -r --delete ./ $TARGET_DIR
  tags:
    - myrunner

This will move your projectfiles in /home/ab12/public_html/

naming your projects as project1 .. projectn, all your projects could use this same .gitlab-ci.yml file.


How does Kubernetes determine which node to use?

In order to avoid this kubernetes reserves resources on the nodes for its daemons. See full explanation and configuration guidelines here


How do we move to a modern CICD approach

You have all the necessary tools for the 2 use cases you mentioned. For application development, it looks like you have something already set up.  I'm assuming, for time's sake, you have automated both code quality and security scanning of your application and that is all working.  If you haven't already started packaging your applications into helm to deploy into kubernetes I suggest you do that.  What your looking to do is automate your deployments of pre-built applications and infrastructure into EKS or cloud (aws).

To deploy applications onto your EKS cluster terraform allows you to deploy via helm.  So your Jenkins pipeline can deploy your terraform infrastructure which utilize the providers for kubernetes, eks, helm, AWS and more to provision and configure your infrastructure and applications. There is a separate pipeline usually for custom apps, pre-built apps, and infrastructure.  Each has it's own quality, security, and resiliency checks needed so you may need some additional tools as you go forward.

For example here is a link to the Hashicorps website on using terraform in automated pipelines.  Note it's considered advanced https://learn.hashicorp.com/terraform/development/running-terraform-in-automation

At this point I'm not sure what you're using Ansible for as that's not necessary for EKS and it's services but if you still need it in parallel to terraform there are plugins to Jenkins https://plugins.jenkins.io/ansible/.   There's also one for terraform https://plugins.jenkins.io/terraform//

So in short, no need to start adding tools but exploring the ones you have.  The way to look at it is this:

Task Runner and orchestrator for CI/CD: Jenkins

this will orchestrate all CI/CD

Application deployment: Helm

this would deploy and version all applications
applications can be deployed via jenkins or terraform

Infrastructure deployment: Terraform

initial testing done manually but Jenkins can run terraform to deploy infrastructure and base services.


Jenkins - sending email after build with the build version as a subject

I found a way to set environmental variables globally from a file!
I'm using envInject for loading variables from a file.

For example, lets say I have a file that is called "versionNumber.txt" and that it's created during the build process. Maybe I update it in every build, or any other way. then I create a prosFile that can be identified by the envInject, by running a batch script:

set /p VERSION_NUM=<versionNumber.txt

Let's assume my version number was 1.5.5.10. so now my VERSION_NUM equals that string.
now VERSION_NUM has the value that I want. But it's local, it won't help me. so I rewrite it to a file in the following format (also, using a batch script):

echo VERSION_NUM=%VERSION_NUM% > C:\Temp\prosFile

now I have a file that contains the following content:

VERSION_NUM=1.5.5.10

And now I just need to load the file using the envInject plugin:

And now I have an environmental variable named VERSION_NUM that can be used anywhere in Jenkins. But I wanted to make it the subject of my email, so I'll simply do (in windows, in linux it might be different):

That's it!


How to dynamically create digital ocean droplets without password change?

After more tests, I found out that in order to do this, I need to use an ssh key with the API. The password change phase is bypassed after this.