Setting up a new Drutopia site

Agaric manages a large number of Drupal, primarily Drutopia sites.

Please see the Drutopia Platform README for an overview of hosting and deploying Drutopia sites If you won’t be deploying, skip overall setup.

Following this guide requires a working DDEV installation.

Overall setup

Strongly recommended to set up locally like this:

mkdir -p ~/Projects/drutopia-platform
cd ~/Projects/drutopia-platform
git clone git@gitlab.com:drutopia-platform/build_source.git
git clone git@gitlab.com:drutopia-platform/drutopia_host.git
cd drutopia_host
git clone git@gitlab.com:drutopia-platform/build_artifacts.git
git clone git@gitlab.com:drutopia-platform/hosting_private.git

Note

These last two repositories are private to Agaric and others maintaining the Drutopia platform, so if you’re following along at home you’ll have to create your own analogs, per the documentation.

Commands for copying throughout will assume this above setup.

Create a new site project

Ordinarily this will live under https://gitlab.com/agaric/sites (for Agaric clients) or https://gitlab.com/drutopia-platform/sites (for direct Drutopia platform members).

Copy the composer.json, composer.lock, and .gitignore files used by the appropriate build source, such as the default build source for the Drutopia Platform.

Replace “example” below with name of the site, usually derived from the main domain name, for instance example-com:

MY_SITE="example"
mkdir -p ~/Projects/agaric/sites/$MY_SITE
cd ~/Projects/agaric/sites/$MY_SITE

Once you have created and are in this directory, whether agaric/sites or drutopia-platform/sites or wherever you want your project to live within the GitLab namespace, you can copy-paste these commands for a quick start:

wget https://gitlab.com/drutopia-platform/build_source/-/raw/master/composer.json
wget https://gitlab.com/drutopia-platform/build_source/-/raw/master/composer.lock
wget https://gitlab.com/drutopia-platform/build_source/-/raw/master/.gitignore
ddev config --docroot=web --project-type=drupal8 --webserver-type=apache-fpm --mariadb-version=10.1 --php-version=7.4 --xdebug-enabled=true --use-dns-when-possible=false --create-docroot
ddev start
ddev composer install

Note

Webserver, PHP, and MySQL versions and types are selected here to match those used on Elizabeth and should be adjusted to match your live environment, including double-checking that they are still valid for elizabeth.mayfirst.org. Please update this documentation if it changes! The last two options (enabling Xdebug by default and not using DNS for your local site) are optional at the developer’s preference. Additional ddev options can be found at ddev.readthedocs.io/en/latest/users/cli-usage.

Setting up database and configuration for the live instance

In order to get a configuration that has the proper site key, it is easiest to first deploy the site to the eventual live location, and sync that database locally.

If you are creating a specialized build of Drutopia, you will have to add that to the host vars, and build that prior to deploying the site. ahoy vars-edit and ahoy deploy-build <build_target> are used for this. Note that new builds should be added ONLY as absolutely required. Configuration, and themes should be leveraged as much as possible prior to resorting to a new build. If additional/different modules are required, a new build is required - do NOT add them to build_source except when they are known to be required for ALL Drutopia basic sites.

Create a new site (member entry) per instructions in Drutopia hosting. The simplest method is to use ahoy new-site <member-slug> and follow it’s output to get started. Then use ahoy deploy-site <member-instance> to deploy one.

Configure drush aliases

The drush site aliases file can be used to provide easy access to the live/test instances of a site. From the root of your project directory (e.g. agaric/sites/example/), you may create one with:

MY_SITE="example-com"
SERVER="drutopia.org"
mkdir -p drush/sites/
cat << EOF > drush/sites/self.site.yml
live:
  host: ${SERVER}
  paths:
    drush-script: /home/${MY_SITE//-/_}_live/site/vendor/bin/drush
  root: /home/${MY_SITE//-/_}_live/site/web
  uri: 'https://${MY_SITE}-live.drutopia.org/'
  user: ${MY_SITE//-/_}_live
test:
  host: ${SERVER}
  paths:
    drush-script: /home/${MY_SITE//-/_}_test/site/vendor/bin/drush
  root: /home/${MY_SITE//-/_}_test/site/web
  uri: 'https://${MY_SITE}-test.drutopia.org/'
  user: ${MY_SITE//-/_}_test
EOF

This will create a self.site.yml using the expected pattern of “site_name_INSTANCE” (e.g. example_com_live for the example-com live instance). Supply the URL form of the site name for the MY_SITE variable (i.e. with dashes, rather than underscores).

Syncing, and setting up configuration

Drutopia releases will expect the configuration in $project_root/config/sync. Be sure to set the appropriate variable in settings.php for it to be stored/retrieved from there. Note not to use settings.ddev.php, as this will be generated during ddev start:

$settings['config_sync_directory'] = '../config/sync';

While in web/sites/default/settings.php also prevent site administrators being told they can install new modules when they can’t and add:

$conf['allow_authorize_operations'] = FALSE;

(All this should be updated to use a distribution-wide settings.php when we have that.)

Once you also have a working Drush installation and a live instance, you can then aquire and export the initial configuration with:

ddev . drush sql-sync @live @self
ddev . drush -y cex

Similarly, you can get live data and files down to your local development environment any time with:

ddev . drush -y sql-dump > data/paranoia.sql
ddev . drush -y sql-drop
ddev . drush -y sql-sync @live @self
ddev . drush -y rsync @live:%files @self:%files

And get the latest code for your distribution (here presuming build_source) like this:

wget https://gitlab.com/drutopia-platform/build_source/-/raw/master/composer.lock
ddev composer install

Or update the build source yourself, if you’ve got it downloaded locally per overall setup above:

ddev composer update
cp composer.json ~/Projects/drutopia-platform/build_source/
cp composer.lock ~/Projects/drutopia-platform/build_source/
cd ~/Projects/drutopia-platform/build_source/
git add -p composer.json
# Make certain no changes to `composer.json` that shouldn't be shared by every site
git commit -m "Add x module or patch for z"
git add -p composer.lock
git commit -m "Update composer lock file"
git push
cd -

Finalize hosting setup

HTTPS Certificates

First, ensure that all domain names are pointed to the new location. Deploy the site with the appropriate server_hostname, and server_aliases already set. Note that if canonical_redirect: true you should use the default <member>-live.drutopia.org for server_hostname until the certificate is provisioned.

Currently, acquiring the certificates must be peformed directly on the server (as root) by running: certbot certonly --webroot -w /home/<member>/site/web -d example.org -d www.example.org -d example-live.drutopia.org

Adjust the vault configuration to include the path to the newly generated cert and key values in the SSL settings for the member (site_ssl_cert_path and site_ssl_key_path). If there are no further changes, an ahoy deploy-site-minimal <member> deployment is enough to activate the certificates (if host names have been added, a normal deploy is necessary for them to reach Drupal’s setting for allowed hostnames).

Reconfigure For Future Deployments

Once the initial site has been installed, be sure to update the vault parameters, as appropriate:

config_import: true
drupal_install: False (or else remove this line)

Potential Traps

If configuration is not importing: log in as root and delete the file /home/<member>/site/CUSTOM-VERSION and run the config-forced install option again - this tells the activate script to repopulate the config folder from the custom source.

See this issue for the latest status on this problem.