Skip to content
14 changes: 14 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

* **playbooks/setup_numishare**: New playbook that wires up the full Numishare stack (`apps`, `apache_solr`, `apache_tomcat`, `numishare`, `existdb`, `orbeon_forms`) in the correct order, including the Solr-on-Numishare specifics (`apache_solr__security_manager_enabled: false`, `apache_solr__dump_cores: ['numishare']`, OS-level dependencies via `apache_solr__apps__apps__dependent_var`)
* **role:orbeon_forms**: New role to deploy [Orbeon Forms](https://www.orbeon.com/) Community Edition into an existing Tomcat for the Numishare stack. Extracts the WAR exploded, rewrites `log4j2.xml` to absolute log paths under `/var/log/tomcat` (Orbeon's relative `../logs/` resolves to `/usr/share/logs/` on RHEL 10 Tomcat and silently disables every FileAppender), deploys `Catalina/localhost/orbeon.xml` with `allowLinking="true"`, wires Numishare's `apps/` symlink and XQJ libraries, ships a Numishare-branded favicon at the legacy `/ops/images/orbeon-icon-16.{ico,png}` path (Orbeon 2023.1 dropped the upstream file), and injects managed Numishare blocks into `properties-local.xml` and `web.xml` (epilogue theme, container auth, security-constraint for `/numishare/admin/*`, `/themes/*` default-servlet mapping, longer session timeout)
* **role:numishare**: New role for [Numishare](https://github.com/ewg118/numishare), the open-source numismatic-collection platform. Clones the repository, deploys `exist-config.xml`, wires the Solr `numishare` core (notifies a Solr restart so the core is loaded), exposes Numishare's bundled UI as the `default` theme via a `/opt/themes/default` symlink, and supports custom themes via `numishare__themes__*` (combined-var pattern, source can be `git_url` + optional `git_version`/`git_update`, or `tarball_url` + optional `tarball_strip_components`)
* **role:existdb**: New role to install and operate eXist-db 6.x. Extracts the upstream `unix.tar.bz2` to `/opt/existdb`, creates the system user/group, shifts the Jetty HTTP/HTTPS ports off `8080`/`8443` (so eXist-db can coexist with Tomcat or Wildfly on the same host), redirects logs to `/var/log/existdb`, sets the admin password on first install only (marker-file gated), points `client.properties` at `127.0.0.1` (eXist-db's Jetty binds IPv4 only), and ships a `mariadb-dump`-style `existdb-dump.service` + `.timer`. README documents the matching `bin/restore.sh` invocation
* **role:apache_tomcat**: Add support for Tomcat 10.1 on RHEL 10. New `etc/tomcat/10.1-{server,context,logging.properties,tomcat-users}.xml.j2` and `etc/sysconfig/10.1-tomcat.j2` templates clone the existing 9.0 set; the `tomcat__installed_version` lookup picks them up automatically. Existing 9.0 deployments are unchanged
* **role:apache_tomcat**: Promote `apache_tomcat__env_xms` / `__env_xmx` / `__env_xx` from inline template defaults to first-class entries in `defaults/main.yml` (defaults `'1024M'` / `'1024M'` / `'+UseParallelGC'`). User-visible behavior is unchanged; users who want to tune the Tomcat heap from their inventory now have documented variables instead of having to chase the `| d(...)` fallback inside the `9.0-tomcat.j2` / `10.1-tomcat.j2` sysconfig templates
* **role:apache_solr**: Add `apache_solr__allow_paths` to expose `-Dsolr.allowPaths` from the inventory (default `[]`). Required when Solr 9 must read or write outside `solr.solr.home` — e.g. backup destinations that the new dump pipeline auto-injects, or cores whose data lives elsewhere. The empty default keeps the property unset, matching upstream behavior
* **role:apache_solr**: Add `apache_solr__heap` (default `'512m'`) so users can size the Solr JVM heap from the inventory. Previously the only knob was the upstream-shipped `#SOLR_HEAP="512m"` comment in `solr.in.sh`, leaving Solr permanently on the 512m default with no role-level override
* **role:apache_solr**: Add `apache_solr__security_manager_enabled` (default `true`, matching Solr 9's documented default) so deployments with a `solr.solr.home` outside Solr's permitted paths can disable the Java SecurityManager from the inventory. Required for Numishare, whose Solr core lives under `/opt/numishare/solr-home/` and otherwise hits `access denied ("java.io.FilePermission" ...)` errors
* **role:apache_solr**: Add a `mariadb-dump`-style backup pipeline: `apache-solr-dump.service` (oneshot) + `.timer` snapshot every core listed in `apache_solr__dump_cores` via Solr's `replication?command=backup` endpoint, polling `command=details` until status is `success`. Snapshots land in `apache_solr__dump_directory` (default `/backup/apache-solr-dump`). Wipe-and-refresh per run; retention is the surrounding backup tool's job. Empty `apache_solr__dump_cores` disables the timer (no-op default). The pipeline also adds `apache_solr__dump_directory` to `-Dsolr.allowPaths` automatically (Solr 9 rejects backup `location=` outside `solr.solr.home` with HTTP 400). The parent of `apache_solr__dump_directory` is created as `0o755 root:root` so other dump pipelines (existdb-dump, mariadb-dump) sharing `/backup/` can still traverse into their own subdirs. README documents the restore via `replication?command=restore` plus `restorestatus` polling
* **role:graylog_datanode**: Add optional variable `graylog_datanode__raw`.
* **role:graylog_datanode**: Add optional variables `graylog_datanode__path_repos`, `graylog_datanode__node_search_cache_size` to configure searchable snapshot locations and size of disk-based searchable snapshot cache.
* **role:infomaniak_vm**: Add `keep_port_on_absent` subkey on `infomaniak_vm__networks` entries to preserve the port (and its fixed IP) when the VM is set to `infomaniak_vm__state: 'absent'`, so the same IP can be re-used
Expand All @@ -25,6 +35,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Fixed

* **role:numishare**: Rewrite the two upstream-hardcoded `/usr/local/projects/numishare` paths in the cloned source tree to `numishare__install_dir`. Affects `xforms/admin.xhtml` (pre-fills the "Installation Directory" field on the Add-New-Collection form, persisted into eXist-db per collection) and `script/reindex-collection.php` (`$eXist_config_path` for the CLI reindex). Without this rewrite the form default points at a non-existent path on every fresh install (lfops uses `/opt/numishare`) and the reindex script can't find `exist-config.xml` until manually patched.
* **role:apache_solr**: Fix `No package java-17-openjdk-headless available.` on RHEL 10. Red Hat dropped `java-17-openjdk` from EL10 AppStream (only `java-21-openjdk` and `java-25-openjdk` ship now). The role now picks `java-21-openjdk-headless` for Solr 9.x on EL10 via a new OS-specific `vars/RedHat10.yml`; EL8/9 keep `java-17-openjdk-headless` unchanged. Solr 9.x officially supports Java 11, 17, and 21.
* **role:apache_solr**: Add `become: false` to the `delegate_to: localhost` `get_url` tasks so the role works under ansible-navigator execution environments, where `become` would try to `sudo` inside the EE container without a password.
* **playbooks/freeipa_client, playbooks/freeipa_server**: Set `strategy: 'linear'` explicitly so the playbooks work even when the user's `ansible.cfg` defaults to a strategy that reuses the target Python interpreter (e.g. `mitogen_linear`). The ansible-freeipa modules rely on `ipalib`'s global API singleton and otherwise fail with `API.bootstrap() already called` on the second module call.
* **role:mariadb_server**: Fix MariaDB starting in the `unconfined_service_t` SELinux domain on RHEL 10, which leaves `/var/lib/mysql/mysql.sock` mislabeled and breaks `php-fpm`/`httpd_t` clients (e.g. Icinga Web 2 login: `SQLSTATE[HY000] [2002] Permission denied`). The unit drop-in's `ExecStartPre=-/bin/chcon -t mysqld_exec_t /usr/sbin/mariadbd` workaround for [MDEV-30520](https://jira.mariadb.org/browse/MDEV-30520) cannot relabel the binary on EL10+, where the packaged `mariadb.service` applies `ProtectSystem` that mounts `/usr` read-only inside the service sandbox. The role now sets the `mysqld_exec_t` file context for `/usr/sbin/mariadbd` persistently via `semanage fcontext` + `restorecon` (outside the systemd sandbox) and notifies a restart so the daemon comes up in `mysqld_t`.
* **role:icinga2_master**: Fix `selinux` role failing on RHEL 10 with `SELinux boolean icinga2_can_connect_all is not defined in persistent policy` (and `[Errno 11]` for the other Icinga/Nagios booleans). The `icinga2-selinux` policy module references `nagios_*_plugin_t` types that were moved out of the EL10 base policy into the separate `nagios-selinux` package (EPEL), so without it the `icinga2-selinux` `%post` silently fails and the booleans never appear. The role now installs `nagios-selinux` as a separate pre-install task on RHEL 10 so its `%post` registers the required types before `icinga2-selinux`'s `%post` runs.
Expand All @@ -48,6 +61,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Changed

* **role:apache_solr**: Download Solr 9.x tarballs from `dlcdn.apache.org` first and only fall back to `archive.apache.org` on failure. The Apache CDN serves currently-supported versions at full speed, while the archive server is intentionally throttled (advertises `Vary: Slow,Glacial`) and previously made `get_url` appear to hang for ~30 minutes per run. The fallback keeps older 9.x versions reachable once they've rotated out of the CDN. Solr 8.x stays on `archive.apache.org` (Lucene path; EOL, CDN no longer mirrors it).
* **role:system_update**: Change default of `system_update__update_time` from `'04:00 + 1 days'` to `'04:{{ 59 | random(seed=inventory_hostname) }} + 1 days'`, so updates are spread deterministically across 04:00–04:59 (minute derived from `inventory_hostname`) instead of all hosts firing at 04:00 sharp
* **role:firewall**: Install `nftables` together with `iptables` for `firewall__firewall == "fwbuilder"` on all distros (previously only installed via per-distro task files on Fedora and RHEL 8/9). The redundant `tasks/Fedora.yml`, `tasks/RedHat8.yml` and `tasks/RedHat9.yml` were removed.
* **role:graylog_server**: Update `server.conf` templates to include `telemetry_enabled = false`.
Expand Down
7 changes: 5 additions & 2 deletions COMPATIBILITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ acme_sh | x | x | x | x | x | | x |
alternatives | x | x | x | x | | x | x |
ansible_init | | | | | | | | Fedora 35+
apache_httpd | x | x | x | x | x | | x |
apache_solr | | | x | x | | | |
apache_tomcat | | | x | x | | | |
apache_solr | | | x | x | x | | |
apache_tomcat | | | x | x | x | | |
apps | | | x | x | x | | |
at | | | x | x | | | | Fedora 35
audit | | | x | x | | | |
Expand All @@ -34,6 +34,7 @@ duplicity | | | x | x | x | | | Fe
elastic_agent | | | | x | | | x |
elastic_agent_fleet_server | | | | x | | | x |
elasticsearch | | | x | x | | | x |
existdb | | | | | x | | |
exoscale_vm | | | | | | | | Fedora 35+
fail2ban | | | x | x | | | |
fangfrisch | | | | x | | | |
Expand Down Expand Up @@ -117,10 +118,12 @@ mount | | | x | x | | | |
network | - | - | x | x | | | |
nextcloud | | | x | x | | | |
nfs_client | | | x | x | | | |
numishare | | | | | x | | |
nfs_server | | | x | | | | |
nodejs | | | x | x | | | |
objectstore_backup | | | x | | | | |
opensearch | | | x | | | | |
orbeon_forms | | | | | x | | |
open_vm_tools | | | x | x | | | |
openvpn_server | | | x | x | x | | |
php | x | x | x | x | | | |
Expand Down
12 changes: 12 additions & 0 deletions playbooks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1231,6 +1231,18 @@ Calls the following roles (in order):
* [icinga2_agent](https://github.com/Linuxfabrik/lfops/tree/main/roles/icinga2_agent)


## setup_numishare.yml

Calls the following roles (in order):

* [apps](https://github.com/Linuxfabrik/lfops/tree/main/roles/apps): `setup_numishare__skip_apps`
* [apache_solr](https://github.com/Linuxfabrik/lfops/tree/main/roles/apache_solr): `setup_numishare__skip_apache_solr`, `setup_numishare__apache_solr__skip_injections`
* [apache_tomcat](https://github.com/Linuxfabrik/lfops/tree/main/roles/apache_tomcat): `setup_numishare__skip_apache_tomcat`
* [numishare](https://github.com/Linuxfabrik/lfops/tree/main/roles/numishare): `setup_numishare__skip_numishare`
* [existdb](https://github.com/Linuxfabrik/lfops/tree/main/roles/existdb): `setup_numishare__skip_existdb`
* [orbeon_forms](https://github.com/Linuxfabrik/lfops/tree/main/roles/orbeon_forms): `setup_numishare__skip_orbeon_forms`


## setup_rocketchat.yml

Calls the following roles (in order):
Expand Down
1 change: 1 addition & 0 deletions playbooks/all.yml
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,7 @@
- import_playbook: 'setup_mastodon.yml'
- import_playbook: 'setup_moodle.yml'
- import_playbook: 'setup_nextcloud.yml'
- import_playbook: 'setup_numishare.yml'
- import_playbook: 'setup_rocketchat.yml'
- import_playbook: 'setup_wordpress.yml'
- import_playbook: 'shell.yml'
Expand Down
78 changes: 78 additions & 0 deletions playbooks/setup_numishare.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
- name: 'Playbook linuxfabrik.lfops.setup_numishare'
hosts: 'all'
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this needs to be lfops_setup_numishare

become: true
any_errors_fatal: true
serial: 1
Comment on lines +3 to +5
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't set these unless absolutely necessary


vars:

setup_numishare__apache_solr__skip_injections__internal_var: '{{ setup_numishare__apache_solr__skip_injections | d(setup_numishare__apache_solr__skip_role__internal_var) }}'
setup_numishare__apache_solr__skip_role__internal_var: '{{ setup_numishare__apache_solr__skip_role | d(false) }}'
setup_numishare__apache_tomcat__skip_role__internal_var: '{{ setup_numishare__apache_tomcat__skip_role | d(false) }}'
setup_numishare__apps__skip_role__internal_var: '{{ setup_numishare__apps__skip_role | d(false) }}'
setup_numishare__existdb__skip_role__internal_var: '{{ setup_numishare__existdb__skip_role | d(false) }}'
setup_numishare__numishare__skip_role__internal_var: '{{ setup_numishare__numishare__skip_role | d(false) }}'
setup_numishare__orbeon_forms__skip_role__internal_var: '{{ setup_numishare__orbeon_forms__skip_role | d(false) }}'

pre_tasks:

- ansible.builtin.import_role:
name: 'shared'
tasks_from: 'log-start.yml'
tags:
- 'always'

roles:

# OS-level helpers (java, git, unzip, ...). Reduce to whatever is missing
# by overriding `apps__apps__host_var` in the inventory. apache_solr's OS
# dependencies (bc, lsof, pwgen, tar) are injected here unless the user
# opts out via setup_numishare__apache_solr__skip_injections.
- role: 'linuxfabrik.lfops.apps'
apps__apps__dependent_var: '{{
(not setup_numishare__apache_solr__skip_injections__internal_var) | ternary(apache_solr__apps__apps__dependent_var, [])
}}'
when:
- 'not setup_numishare__apps__skip_role__internal_var'
Comment on lines +27 to +36
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just do this directly in the role itself


# Search backend. Numishare's solr-home lives at /opt/numishare/solr-home
# (outside Solr's default permitted paths), so the Java SecurityManager
# blocks reads with `access denied ("java.io.FilePermission" ...)`. Disable it.
# The `numishare` core is fed from eXist-db and could be rebuilt from there,
# but we still snapshot it via apache-solr-dump for a fast restore path.
- role: 'linuxfabrik.lfops.apache_solr'
apache_solr__security_manager_enabled: false
apache_solr__dump_cores:
- '{{ numishare__solr_core_name | d("numishare") }}'
when:
- 'not setup_numishare__apache_solr__skip_role__internal_var'

# Application server (required by Orbeon Forms). Runs before numishare so
# the `tomcat` user/group exist when numishare chowns its files.
- role: 'linuxfabrik.lfops.apache_tomcat'
when:
- 'not setup_numishare__apache_tomcat__skip_role__internal_var'

# Numishare checkout, exist-config, Solr core wiring, themes dir.
# Runs before existdb/orbeon so its files exist when those reference them.
- role: 'linuxfabrik.lfops.numishare'
when:
- 'not setup_numishare__numishare__skip_role__internal_var'

# XML database backing Numishare.
- role: 'linuxfabrik.lfops.existdb'
when:
- 'not setup_numishare__existdb__skip_role__internal_var'

# Orbeon WAR deployment + Numishare-specific properties / web.xml / themes wiring.
- role: 'linuxfabrik.lfops.orbeon_forms'
when:
- 'not setup_numishare__orbeon_forms__skip_role__internal_var'

post_tasks:

- ansible.builtin.import_role:
name: 'shared'
tasks_from: 'log-end.yml'
tags:
- 'always'
58 changes: 58 additions & 0 deletions roles/apache_solr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -259,6 +259,64 @@ apache_solr__users__host_var:
```


## Backup and Restore

The role can deploy a `mariadb-dump`-style backup pipeline for one or more Solr cores. It uses Solr's `replication?command=backup` endpoint, polls `command=details` until the backup status reports `success`, and writes the snapshot under `apache_solr__dump_directory`. On every run the directory is wiped and refreshed; retention is the responsibility of the surrounding backup tool (Borg, Restic, ...) which snapshots that directory.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove the reference to mariadb


### Optional Backup Variables

`apache_solr__dump_cores`

* List of cores to back up. Empty disables the timer and stops the existing one.
* Type: List of strings.
* Default: `[]`

`apache_solr__dump_directory`

* Where the latest snapshot lands. Owned by `{{ apache_solr__user }}:{{ apache_solr__group }}` so Solr can write into it.
* Type: String.
* Default: `'/backup/apache-solr-dump'`

`apache_solr__dump_on_calendar`

* `OnCalendar=` value for `apache-solr-dump.timer`. Default seeds the minute by `inventory_hostname` so a fleet does not all hit Solr at the same second.
* Type: String.
* Default: `'*-*-* 22:{{ 59 | random(seed=inventory_hostname) }}:00'`

`apache_solr__dump_url`

* Base URL of the Solr instance the dumper hits. Defaults to localhost on the configured port. Override if Solr listens on a UNIX socket or the loopback alias differs.
* Type: String.
* Default: `'http://127.0.0.1:{{ apache_solr__http_bind_port }}/solr'`

### Restoring a Core

1. Stop Solr writes to the target core (route traffic away or stop the service).
2. Identify the snapshot directory written by the dumper:

```bash
ls /backup/apache-solr-dump/snapshot.<core_name>/
```

3. Hit the replication restore endpoint:

```bash
curl 'http://127.0.0.1:8983/solr/<core_name>/replication?command=restore&location=/backup/apache-solr-dump&name=<core_name>'
```

4. Poll until done:

```bash
curl 'http://127.0.0.1:8983/solr/<core_name>/replication?command=restorestatus&wt=json'
```

`status` flips to `success` (or `failed`) when the restore completes.

5. Re-enable traffic.

For Numishare specifically: the `numishare` core is also reproducible from eXist-db (the Numishare publish pipeline rebuilds the index). The Solr snapshot exists to skip the (potentially long) rebuild during disaster recovery, not as the only authoritative source.


## License

[The Unlicense](https://unlicense.org/)
Expand Down
Loading
Loading