User Tools

Site Tools


infra:hscloud

hscloud

VMs are coming any day now ~q3k, A.D. 2018

Our new internal highly-available Infrastructure/Platform-as-a-Service.

This runs in our datacenter (dcr01 on netbox). This is different from our ISP services or internal machines.

Components

Currently hscloud is made up of a Kubernetes cluster named k0.hswaw.net. It runs on the following machines: bc01n01, bc01n02, bc01n03, dcr01s22, dcr01s24. In total we have 328GB of RAM and 144 x86 cores. We also have a half PB of storage on old SAS drives (most of it currently cold), accessible via Ceph (radosgw or Kubernets PersistentVolumes).

Tenants

We are moving services from our old machines into Kubernetes. Amongst other, currently running on the cluster is:

With more to come.

You are also free to host your own personal stuff there within reason. See below for access.

Boston Evacuation Aktion

Here's a list of services that currently live on Boston Packets, but we'd like to migrate to hscloud. Ask on #infra on how to contribute.

Monitoring

Documentation, Getting Access and Usage

Self-documenting in hackdoc (hscloud documentation stored within hscloud): https://hackdoc.hackerspace.pl/doc/codelabs/index.md

Deploy docker image to hscloud

TBD, WIP: You need to describe a deployment of your docker image in the same way as it's done for https://cs.hackerspace.pl/hscloud/-/tree/hswaw/paperless .

Build your docker image by running following command:

docker build --tag registry.k0.hswaw.net/$YOUR_USERNAME/$APP_NAME-$APP_VERSION

Get your login credentials by going to this site https://registry.k0.hswaw.net/ and authenticating via SSO. You should get a command by which you'll login to our docker registry.

Afterwards, push your image by using

docker push registry.k0.hswaw.net/$YOUR_USERNAME/$YOUR_IMAGE_TAG

Commands with example data

docker build --tag registry.k0.hswaw.net/palid/walne-generator:1.0-alpha
docker push registry.k0.hswaw.net/palid/walne-generator:1.0-alpha

Resources/Services

Here is a list of common external/internal services used by apps hosted in hscloud, with guidelines on how to get access to those:

  • Persistent storage/Block storage
    • Use PersistentVolumeClaim in waw-hdd-redundant-3 storage class
  • S3/Object storage
    • Add user object in //cluster/kube/k0.libsonnet:k0.ceph.clients
    • Ask hscloud ops to update
  • CockroachDB
    • Add user object in //cluster/kube/k0.libsonnet:k0.cockroach.clients
    • Ask hscloud ops to update
  • Docker Container Registry
  • DNS
    • *.hackerspace.pl, *.hswaw.net: Ask hscloud ops to create/update relevant DNS entries in ns{1,2}.bytesexual.net, adjust admitomatic config (see below)
    • *: create your own DNS CNAME record pointing at ingress.k0.hswaw.net
    • Adjust //cluster/kube/k0.libsonnet:k0.admitomatic.cfg.proto.allow_domain if you want your domain to be secured against hijacking by other cluster users (*.hackerspace.pl is one such domain)
  • Postgres
    • Use //kube/postgres.libsonnet to create local deployment in app namespace
    • Alternative: ask hscloud ops for a database on blessed high-performance ssd node
  • Redis
    • Use //kube/redis.libsonnet to create local deployment in app namespace
  • SSO (OAuth2/OpenID Connect for HSWAW members authentication)
  • LDAP (only very specific cases, when user/group listing is required - otherwise use SSO)
    • Ask ops to create an LDAP service account
      • Create cn=…,ou=Services,dc=hackerspace,dc=pl in LDAP
      • Add relevant ACL in /etc/openldap/slapd.conf on boston-packets.hackerspace.pl
  • Beyondspace (access to *.waw.hackerspace.pl services from WAN/hscloud)
    • Ask ops to add specific internal domain to beyondspace (//hswaw/machines/customs.hackerspace.pl/beyondspace.nix) and create a service authentication token on customs
  • Mailing (SMTP/IMAP)
    • Ask ops to create local mailing user account
      • Create local service user on boston-packets.hackerspace.pl: useradd -rm SERVICE; passwd SERVICE
      • Create mailbox on boston-packets.hackerspace.pl: mkdir /var/spool/mail/SERVICE; chown SERVICE:mail /var/spool/mail/SERVICE
      • Optionally: add aliases (and/or incoming exec hook) in /etc/mail/aliases
infra/hscloud.txt · Last modified: 2024/07/20 20:23 by radex

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki