Setting up a new Drutopia site

Agaric manages a large number of Drupal, primarily Drutopia sites.

Please see the Drutopia Platform README for an overview of hosting and deploying Drutopia sites If you won’t be deploying, skip overall setup.

Following this guide requires a working DDEV installation. Deploying to Drutopia will require a locally installed composer, as well as ansible. See also the general Git setup if you have not yet.

Overall setup

In line as much as possible with recommended locations, the local setup can be done like this:

mkdir -p ~/Projects/drutopia-platform
cd ~/Projects/drutopia-platform
git clone git@gitlab.com:drutopia-platform/build_source.git
git clone git@gitlab.com:drutopia-platform/drutopia_host.git
cd drutopia_host
git clone git@gitlab.com:drutopia-platform/build_artifacts.git
git clone git@gitlab.com:drutopia-platform/hosting_private.git

Note

These last two repositories are private to Agaric and others maintaining the Drutopia platform on May First’s Elizabeth server, so if you are following along at home you’ll have to create your own analogs, per the documentation.

Commands for copying throughout will assume this above setup.

Create a new site project

For Agaric clients, the online home for the project will usually be at Agaric’s Forgejo project hosting site, first creating an organization (such as example-client, the short name that Forgejo takes first must be treated as a machine name) and then the project (such as example-org) that would live a URL such as https://git.agaric.com/example-client/example-org.

Copy the part of the URL after git.agaric.com so the namespace will be consistent on your local machine and create the directory at ~/Projects. With the examples above it would be mkdir -p ~/Projects/example-client/example-org and cd ~/Projects/example-client/example-org.

Once you have created and are in this directory, wherever you want your project to live within that namespace, you can copy-paste these commands for a quick start:

wget https://gitlab.com/drutopia-platform/build_source/-/raw/main/composer.json
wget https://gitlab.com/drutopia-platform/build_source/-/raw/main/composer.lock
wget https://gitlab.com/drutopia-platform/build_source/-/raw/main/.gitignore
mkdir -p scripts
wget -O scripts/pull.sh https://gitlab.com/drutopia-platform/build_source/-/raw/main/scripts/pull.sh
wget -O scripts/retain-custom-htaccess.sh https://gitlab.com/drutopia-platform/build_source/-/raw/main/scripts/retain-custom-htaccess.sh
chmod +x scripts/pull.sh
chmod +x scripts/retain-custom-htaccess.sh
ddev config --docroot=web --project-type=drupal10 --webserver-type=apache-fpm --database=mariadb:10.8 --php-version=8.1 --create-docroot
ddev start
ddev auth ssh
ddev composer install
git init
git add .
git commit -m "Begin repository based on build_source main"

Note

Webserver, PHP, and MySQL versions and types are selected here to match those used on Elizabeth and should be adjusted to match your live environment, including double-checking that they are still valid for elizabeth.mayfirst.org. Please update this documentation if it changes! The last two options (enabling Xdebug by default and not using DNS for your local site) are optional at the developer’s preference. Additional ddev options can be found at ddev.readthedocs.io/en/latest/users/cli-usage.

Setting up database and configuration for the live instance

In order to get a configuration that has the proper site key, it is easiest to first deploy the site to the eventual live location, and sync that database locally.

If you are creating a specialized build of Drutopia, you will have to add that to the host vars, and build that prior to deploying the site. ahoy vars-edit and ahoy deploy-build <build_target> are used for this.

Note

New builds should be added only as absolutely required. Configuration, and themes should be leveraged as much as possible prior to resorting to a new build. If additional modules are required, consider adding them to build_source, knowing they will be available for all Drutopia SaaS sites.

Create a new site (member entry) per instructions in Drutopia hosting. The simplest method is to use ahoy new-site <member-slug> and follow its output to get started. Then use ahoy deploy-site <member-instance> to deploy one. Because the install using drush site-install that would be run when drupal_install: true is set in the site settings in Drutopia hosting, it is currently recommended to set this to false and use the UI installer or run php docroot/core/scripts/drupal quick-start drutopia --no-interaction when ssh’d into the server.

Configure drush aliases

The drush site aliases file can be used to provide easy access to the live/test instances of a site. From the root of your project directory (e.g. agaric/sites/example/), you may create one with:

MY_SITE="example-com"
SERVER="drutopia.org"
mkdir -p drush/sites/
cat << EOF > drush/sites/self.site.yml
live:
  host: ${SERVER}
  paths:
    drush-script: /home/${MY_SITE//-/_}_live/site/vendor/bin/drush
  root: /home/${MY_SITE//-/_}_live/site/web
  uri: 'https://${MY_SITE}-live.drutopia.org/'
  user: ${MY_SITE//-/_}_live
test:
  host: ${SERVER}
  paths:
    drush-script: /home/${MY_SITE//-/_}_test/site/vendor/bin/drush
  root: /home/${MY_SITE//-/_}_test/site/web
  uri: 'https://${MY_SITE}-test.drutopia.org/'
  user: ${MY_SITE//-/_}_test
EOF

This will create a self.site.yml using the expected pattern of site_name_INSTANCE (e.g. example_com_live for the example-com live instance). Supply the URL form of the site name for the MY_SITE variable (i.e. with dashes, rather than underscores).

Syncing, and setting up configuration

Our standard is to place configuration exports in $project_root/config. Set the this in settings.php:

$settings['config_sync_directory'] = '../config';

While in web/sites/default/settings.php also prevent site administrators being told they can install new modules when they can’t and add:

$settings['allow_authorize_operations'] = FALSE;

This settings.php file is only being used for local development. Drutopia host is responsible for creating all settings during deployment.

(All this should be updated to use a distribution-wide settings.php when we have that. Until the, use git add -f web/sites/default/settings.php to include this in your repo)

Note

Do not to use settings.ddev.php, as this will be re-generated during ddev start.

Once you also have a working Drush installation and a live instance, you can then aquire and export the initial configuration with:

ddev drush sql-sync @live @self
ddev drush -y cex

Similarly, you can get live data and files down to your local development environment any time with:

ddev drush -y sql-dump > data/paranoia.sql
ddev drush -y sql-drop
ddev drush -y sql-sync @live @self
ddev drush -y rsync @live:%files @self:%files

And get the latest code for your distribution (here presuming build_source) like this:

wget https://gitlab.com/drutopia-platform/build_source/-/raw/master/composer.lock
ddev composer install

Or update the build source yourself, if you’ve got it downloaded locally per overall setup above:

ddev composer update
cp composer.json ~/Projects/drutopia-platform/build_source/
cp composer.lock ~/Projects/drutopia-platform/build_source/
cd ~/Projects/drutopia-platform/build_source/
git add -p composer.json
# IMPORTANT: Make no changes to `composer.json` that shouldn't be shared by every site
git commit -m "Add x module or patch for z"
git add -p composer.lock
git commit -m "Update composer lock file"
git push
cd -

Finalize hosting setup

Domain name

Configure the domain name to point its nameservers at:

a.ns.mayfirst.org
b.ns.mayfirst.org

Then set up a new hosting order. Give the username the same name as your repository (“example-com” as used throughout this documentation).

Then under DNS change the a record for your full domain to IP Address 162.247.75.218 and make any additional CNames you want (at least www. for your domain, for instance “www.example.com” to go to Server Name of your domain, for instance “example.com”). You can delete the value for www. that is already there.

HTTPS Certificates

First, ensure that all domain names are pointed to the new location. Deploy the site with the appropriate server_hostname, and server_aliases already set. Note that if canonical_redirect: true you should use the default <member>-live.drutopia.org for server_hostname until the certificate is provisioned.

Currently, acquiring the certificates must be peformed directly on the server (as root) by running: certbot certonly --webroot -w /home/<member>/site/web -d example.org -d www.example.org -d example-live.drutopia.org

Adjust the vault configuration to include the path to the newly generated cert and key values in the SSL settings for the member (site_ssl_cert_path and site_ssl_key_path). If there are no further changes, an ahoy deploy-site-minimal <member> deployment is enough to activate the certificates (if host names have been added, a normal deploy is necessary for them to reach Drupal’s setting for allowed hostnames).

Reconfigure For Future Deployments

Once the initial site has been installed, be sure to update the vault parameters, as appropriate:

config_import: true
drupal_install: False (or else remove this line)

Potential Traps

If configuration is not importing: log in as root and delete the file /home/<member>/site/CUSTOM-VERSION and run the config-forced install option again - this tells the activate script to repopulate the config folder from the custom source.

See this issue for the latest status on this problem.