From ed69f73d4a6f15a69bceb69ce9209930a01f067a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Stephan=20D=C3=B6rfler?= Date: Mon, 10 Feb 2020 07:58:45 +0100 Subject: [PATCH] added deployment config and pipeline --- .drone.yml | 19 ++++ .../{hello-world => 001-hello-world}/index.md | 0 .../{deployment => 002-deployment}/index.md | 0 content/blog/003-writing-articles/index.md | 91 +++++++++++++++++++ gatsby-browser.js | 1 + gatsby-config.js | 2 +- src/styles/global.css | 4 - 7 files changed, 112 insertions(+), 5 deletions(-) create mode 100644 .drone.yml rename content/blog/{hello-world => 001-hello-world}/index.md (100%) rename content/blog/{deployment => 002-deployment}/index.md (100%) create mode 100644 content/blog/003-writing-articles/index.md diff --git a/.drone.yml b/.drone.yml new file mode 100644 index 0000000..9603d76 --- /dev/null +++ b/.drone.yml @@ -0,0 +1,19 @@ +kind: pipeline +type: docker +name: default + +steps: +- name: build + image: node:alpine + commands: + - npx gatsby build + +- name: docker + image: plugins/docker + settings: + username: + password: + repo: registry.while-false.de/blog + tags: + - 'latest' + - '1.3.0' \ No newline at end of file diff --git a/content/blog/hello-world/index.md b/content/blog/001-hello-world/index.md similarity index 100% rename from content/blog/hello-world/index.md rename to content/blog/001-hello-world/index.md diff --git a/content/blog/deployment/index.md b/content/blog/002-deployment/index.md similarity index 100% rename from content/blog/deployment/index.md rename to content/blog/002-deployment/index.md diff --git a/content/blog/003-writing-articles/index.md b/content/blog/003-writing-articles/index.md new file mode 100644 index 0000000..bef1186 --- /dev/null +++ b/content/blog/003-writing-articles/index.md @@ -0,0 +1,91 @@ +--- +title: Adding Content +date: "2020-01-27T15:38:26.882Z" +description: Bring the content online +--- + +# Writing + +This is a blog. So first of all to add new content to it, I write stuff. For now the topics originate from the development and deployment of the blog itself. + +I write all of it in `markdown` (files ending in `.md`), organized in a single folder per post. On build, these markdown files are automatically converterd to `HTML` and after deployment thats what your browser shows you. Your browser retrieves the content from a webserver, in my case that's a `nginx` from inside a docker container (actually there's another `nginx` in a docker container inbetween acting as a reverse proxy). As I described in the last post, the docker container is created from the default gatsby base image, which copies the built (optimized) contents of the blog into the docker container and configures a `nginx` to serve it. + +For now this is a manual process, which means I usually write the posts in Visual Studio Code on my PC, commit them in my `git` repository and then push them to my [personal git server](https://code.while-false.de). The same server also hosts the blog and some other things (to be explained in future posts) all separated in individual docker containers. I then + +1. log onto my server via ssh `ssh stephan@while-false.de` +2. pull the latest version from said git repository `git pull` +3. build the blog `npx gatsby build` +4. build a new version of the docker container `docker build -t while-false/blog` +5. swap the (old) running version of the container with the newly built one. `docker stop blog && docker rm blog && docker run --name blog while-false/blog` + +(Some of the commands actually need more parameters for production use, but are not relevant for the concept) + +All steps on the server are handled from the terminal and typing the same commands over and over on each new post is rather unconfortable. + +# Automation - Planning + +Whenever you do the same thing often, automation comes to mind. So the steps on the server are to be automated. I have some ideas for that. + +The relevant event I want to react on is the change in the blog. I consider a change as relevant, when a new commit happens on the `master` branch of the `git` repository of the blog. Luckily, `git` has a builtin concept for reacting on events on the server, called "server-side hooks". In my case the `git` server is an instance of `gitea`, so I looked up server side hooks in [the gitea documentation](https://docs.gitea.io/en-us/webhooks/). I quickly found the hook I needed: + +So I can make `git` notify some other component of each change. Now I need something to listen to these notifications and then execute the update-steps automatically. + +My first intention was to build that component myself. I know all the commands to execute (see above). I would just need some preparation: + +* Have something listen on HTTP for the call of the git hook +* Then pull the latest commit from the git server (it should be included in the call of the hook) +* Oh wait, I need to authenticate the component first. Maybe with ssh keys? +* Then install the npm dependencies. But first make sure node.js is really installed +* Build the gatsby.js project. But first make sure gatsby.js is really installed +* Build the docker image. But first make sure docker is really installed +* Push the docker image to my private docker repository. Another authentication required +* On the host exchange the currently running docker container with the newly built one + +Around that time of planning I decided this isn't the way to go. At my job I rely heavily on [Azure DevOps](https://azure.microsoft.com/en-us/services/devops/) which conveniently covers all these tasks and requirements. But for my private free-time-projects I imposed the restriction on myself to run everything on my own server(s). But until now I only looked at the furthest cases on the automation spectrum: doing everything myself and have everything done by Microsoft in the cloud. I decided that the truth probably lies somewhere inbetween (as so often in life). I then looked at self-hosted CI/CD ("Continous Integration"/"Continous Delivery") systems. + +[Drone](https://drone.io) caught my eye, as it has full docker support, is open source and can have multiple, distributed workers. Perfect, I finally get to use the "sandbox" VPS I rent which just accumulates virtual dust. After reading the documentation, the setup was fairly easy. + +First I spun up a docker container for drone, `drone/drone` on Docker-Hub. It has lots of required arguments so I will only describe some of the configuration: + +* The gitea server I want to depend upon (other git servers can also be used) +* oAuth2 credentials (clientId and clientSecret) for an application registered on the gitea server +* A self-generated secret to use for connection of the workers +* Hostname and protocol (`https`) of the drone server itself + +As mentioned above I also wanted to use another machine as a worker for drone. Luckily, theres also a docker image for that: `drone/drone-runner-docker`. The worker also has some (similar) parameters: + +* The hostname and protocol of the drone server (see above) +* The shared secret to connect the worker to the ci server (see above) +* The hostname of the worker +* The parallelism of the worker + +The worker then connected to the drone server. In the UI of the drone server I could then log in using my account from the gitea server (yay, oAuth!) and immedeatly see the repository for the blog. There are only a few options for the repository, because the build pipeline itself is defined as code in a file `.drone.yml`. For my case with the gatsby blog I use the following pipeline: + +```yaml +kind: pipeline +type: docker +name: default + +steps: +- name: build + image: node:alpine + commands: + - npx gatsby build + +- name: docker + image: plugins/docker + settings: + username: *** + password: *** + repo: registry.while-false.de/blog + tags: + - 'latest' + - '1.3.0' +``` + +For now I only require two steps: + +1. build the gatsby project +2. build the new docker image and push it to the registry + +Then, the last required step is to update the running container to the new version. The event on which to react would be the upload to the registry. diff --git a/gatsby-browser.js b/gatsby-browser.js index e0d3fdb..388af4d 100644 --- a/gatsby-browser.js +++ b/gatsby-browser.js @@ -3,3 +3,4 @@ import "typeface-montserrat" import "typeface-merriweather" import "./src/styles/global.css"; +import "prismjs/themes/prism-solarizedlight.css"; diff --git a/gatsby-config.js b/gatsby-config.js index e736568..e6a9550 100644 --- a/gatsby-config.js +++ b/gatsby-config.js @@ -2,7 +2,7 @@ module.exports = { siteMetadata: { title: `While False Blog`, author: `Stephan Dörfler`, - description: `My try on a self-defined blog.`, + description: `Self-built blog based on gatsby.`, siteUrl: `https://blog.while-false.de/`, }, plugins: [ diff --git a/src/styles/global.css b/src/styles/global.css index 1556ef7..b1864e8 100644 --- a/src/styles/global.css +++ b/src/styles/global.css @@ -1,7 +1,3 @@ body { background-color: #e3dcc2; -} - -code { - background-color: lightgray; } \ No newline at end of file