Selfhosted Infrastructure Any%
A Co-op Cloud speedrun attempt livestream! Brought to you by by libre software.
It’d been so long since the last Co-op Cloud demo video that, since then, our core command-line tool abra
had been completely rewritten (from Bash to Golang), and the entire Co-op Cloud project moved from an in-house Autonomic production to a participatory, democratic Co-op Cloud federation. Definitely time for an update!
So, Co-op Cloud nerd @3wc
picked up the fine tradition of speedrunning for a “Selfhosted Infrastructure Any%” livestream – attempting to beat their personal best time of 46m17s to deploy chat (Matrix / Fluffychat), microblogging (Hometown), and file-/calendar-sharing (Nextcloud) behind single sign-on (Keycloak).
Before we dive into the write-up, mega-thanks to audience member @yala
for putting together an excellent write-up during the stream, including some insightful questions. And, of course, to everyone who came along, and everyone who helped signal-boost the announcement beforehand!
Concept
The integrated set-up of Keycloak/Matrix/Hometown/Nextcloud is a running start for a small-to-medium group looking for alternatives to corporate spyware. It comprises solid alternatives to Dropbox/Google Drive, Twitter, and Slack/Discord/Whatsapp. And, the entire stack can be hosted on a cheap server (or even an old laptop or desktop computer), and the services can be accessed using reasonably slick mobile and desktop apps.
Single sign-on means that in some ways, this set-up is even more usable, even for non-technical users, than the corporate equivalents. And, highlighting where Co-op Cloud can help, getting this combination of tools running manually without Co-op Cloud (that is, following the separate set-up instructions for each platform) is a non-trivial technical challenge.
The choice of apps is primarily geared towards working groups of 3-200 people, and the focus is on internal coördination platforms rather than publication.
Preparation
To try and simulate an experience of setting up these apps for the first time, and as a nod to the purist conditions popular with videogame speedruns, the demo starts with a blank Debian Docker image (docker run debian
) and a blank Debian 12 server. (All commands below are run inside that container, unless specified otherwise).
To avoid needing to screenshare DNS and server configuration, there are DNS records for speedrun.coopcloud.tech
and *.speedrun.coopcloud.tech
set up, and a pre-generated SSH key has permission to log into the server as root.
Howto / script
1 Local setup
apt update && apt install vim curl wget openssh-client -y
export EDITOR=vim
(The recommended installation instructions for abra
require both curl
and wget
for some reason. An issue has been opened to figure that out: #591
)
The next step is to copy-paste this from the abra
installation instructions:
curl https://install.abra.coopcloud.tech | bash
An audience member commented that
curl X | bash
is not amazing security, and indeed downloading the latestabra
release binary manually, and comparing the file checksums, would be more secure – patches todocs.coopcloud.tech
extremely welcome! See more on#593
.
Then, running abra autocomplete bash
shows the necessary commands to set us up some autocompletion for abra
commands.
2. Server set-up
The next step, on the virtual server, is to run the “Server set-up” section of the Co-op Cloud docs:
ssh speedrun.coopcloud.tech
# docker install convenience script
wget -O- https://get.docker.com | bash
# setup swarm
docker swarm init
docker network create -d overlay proxy
(Skipping adding the current user to the docker
group, because root
already has permission to access the Docker daemon)
As above,
wget ... | bash
is a bit wacky, this is the Docker-recommended installation method but there are others – patches welcome!
Lastly, back on the local machine, abra app server add speedrun.coopcloud.tech
3. Traefik
abra app new traefik
abra app config traefik.speedrun.coopcloud.tech
abra app deploy traefik.speedrun.coopcloud.tech -c -n
(-c
to skip waiting for the container to come up, and -n
to skip the confirmation prompt)
4. Keycloak
abra app new keycloak --secrets
abra app config keycloak.speedrun.coopcloud.tech
abra app deploy keycloak.speedrun.coopcloud.tech -c -n
Then the Keycloak interface is available at https://keycloak.speedrun.coopcloud.tech
, necessary to set up the OpenID Connect “clients” for single sign-on for the other apps.
5. Matrix
abra app new matrix-synapse
Meanwhile, by this point Keycloak is alive, and ready to create a new “client”. Doing so requires specifying a “client ID”, and generates a new “client secret”, both of which get added:
abra app config matrix-synapse.speedrun.coopcloud.tech --secrets
# fill in SSO details
abra app secret insert matrix-synapse.speedrun.coopcloud.tech keycloak_client_secret v1 (value)
abra app deploy matrix-synapse.speedrun.coopcloud.tech -n -c
6. Fluffychat
abra app new fluffychat
abra app config fluffychat.speedrun.coopcloud.tech -n -c
7. Hometown
abra app new hometown
Again, a new Keycloak “client” is needed, giving us a client secret and client ID.
Additionally (at least, for now), it’s necessary to do some slightly cursed hacking on the server:
docker run -it git.coopcloud.tech/coop-cloud-chaos-patchs/hometown:v4.0.15-hometown-1.1.1 bash
/usr/local/bin/docker-entrypoint.sh bundle exec rake secret # once for otp_secret
/usr/local/bin/docker-entrypoint.sh bundle exec rake secret # then a second time for secret_key
/usr/local/bin/docker-entrypoint.sh bundle exec rake mastodon:webpush:generate_vapid_key
Then manually insert the secrets, generate the remaining ones, and edit in the VAPID_PUBLIC_KEY
and the other SSO details:
abra app secret insert hometown.speedrun.coopcloud.tech vapid_private_key v1 <value>
abra app secret insert hometown.speedrun.coopcloud.tech otp_secret v1 <value>
abra app secret insert hometown.speedrun.coopcloud.tech secret_key v1 <value>
abra app config hometown.speedrun.coopcloud.tech
# set OIDC_* settings, and VAPID_PUBLIC_KEY
Now, finally, the app can be deployed!
abra app deploy hometown.speedrun.coopcloud.tech -n -c
8. Nextcloud
abra app new nextcloud --secrets
abra app deploy nextcloud.speedrun.coopcloud.tech
Annoyingly, Nextcloud needs stuff done in the web interface, and in the PHP config file, for SSO. So, with another new Keycloak client set up, the next step is logging into https://nextcloud.speedrun.coopcloud.tech, going to “Apps”, installing “OIDC login by pulsejet”, and then editing the config file:
abra app run nextcloud.speedrun.coopcloud.tech
apt update && apt install vim
vim config/config.php
# fill in from https://git.coopcloud.tech/coop-cloud/nextcloud#how-do-i-integrate-with-keycloak-sso
(in the demo, this was done from the server, but the above approach is probably conceptually simpler)
Closing thoughts
Even though no records got broken, this was fun to make, and definitely a better way to prepare for an in-person demo than silent solo-hacking. It’s also nice to get a little “this is how Co-op Cloud is currently looking” demo video out of it.
If you have ideas for future streams, hit us up on Mastodon (@coopcloud@social.coop) or dive into our Matrix (#coopcloud:autonomic.zone). Otherwise, follow us directly on owncast (@cast@cast.coopcloud.tech
) and/or peertube (@coopcloud@tv.undersco.re) to hear about future streams and videos when they happen!