Below is an outline of how the various pieces of the PASS infrastructure will be coordinated into a release. We will start with non-production-ready version using pass-docker and in particular one based on GitHub Actions via Self-Hosted Runner.
Once branch 178-docker-compose-gh-actions
is merged, this documentation should be updated to
point back to the main
branch.
As different environments (cd, nightly, dev, demo, pre-production, production) are introduced this documentation will be updated to reflect those differences (for example cd, nightly, dev, demo will all use a fake nihms FTP server).
The docker containers should be migrated based on eclipse-pass dependencies.
The PASS application will be responsible for
Note that the pass-app
currently refers to the pass-docker repo.
The recommendation is that pass-docker
be renamed (and enhanced) to
capture using infrastructure as code) all configurable environments.
Here is a summary of the actions and their current triggers / implementation status (note not all actions have been fully automated).
Action | Trigger | Note |
---|---|---|
deploy_demo | GH Action Button | In development |
update_core_versions | Manual | Manually build / publish containers, update pass-docker manifests |
update_datastores | Manual | Same as update_core_versions |
update_project | Manual | Update pass-docker manifests |
The individual projects will be responsible for
The artefacts will be stored in a consistent manner so the application can correct grab the appropriate versions when orchestrating a release.
Action | Trigger | Note |
---|---|---|
test | GH PRs | Mostly in place for all projects |
build | Manual | Working towards GH actions |
notify | Manual | Notify via email/slack/carrier pigeon of update |
The PASS project can be viewed as a decomposition of
Each decomposition will have different levels of automation and different release cycles.
The following core pieces of infrastructure are required for the PASS Application. These pieces are more stable and will not be updated frequently.
Name | Based On | Notes |
---|---|---|
fcrepo | tomcat:8.5.40-jre8-alpine docker | Old version (4.7.5) and relies heavily on OAPASS instead of eclipse-pass |
postgres | postgres:10.3-alpine docker | Underlying DB for fcrepo |
activemq | openjdk:8-jre-alpine docker | Version 5.15.11 |
httpd-proxy | centos docker | Orchestration between services |
elasticsearch | elasticsearch/elasticsearch-oss 6.2.3 docker | Security vulnerability to patch |
oapass/docker-mailserver | Should be updated to eclipse-pass/pass-docker-mailserver |
The following third party data stores are configured / configurable within PASS. These pieces are more stable and will not be updated frequently.
Name | Based On | Notes |
---|---|---|
nihms ftpserver | stilliard/pure-ftpd docker | Sample Nimhs FTP Server |
dspace | jetty:9.4.7-jre8-alpine docker | Sample DSpace database |
The following core pieces manage authentication. These pieces are more stable and will not be updated frequently.
Name | Based On | Notes |
---|---|---|
idp | unicon/shibboleth-idp docker | Based on shibboleth and configurable for common, harvard and jhu |
ldap | centos docker | Configured for fakek accounts on common, harvard and jhu |
shibboleth-sp | unicon/shibboleth-sp docker | Core shibboleth service provider |
authz | openjdk:8u212-jre-alpine3.9 docker | Managed via oapass.org |
The following projects will manage building their own containers.
Name | Installation | Notes |
---|---|---|
pass-ui | docker | Relies on OA-PASS/pass-ember and should be updated to eclipse-pass/pass-ui |
pass-ui-static | docker | Relies on OA-PASS should be updated to eclipse-pass |
indexer | openjdk:8u212-jre-alpine3.9 docker | Relies on OA-PASS should be updated to eclipse-pass |
assets | alpine:3.7 | Based on minimal assets data.tar within pass-docker |
deposit-services | pass-package-providers docker | Based on oapass/deposit-services-providers and should be moved to eclipse-pass/pass-package-providers Configurable for JHU or Harvard |
notification-services | notification-services/0.1.0-3.4 | Update to to eclipse-pass/pass-notification-services |
schemaservice | oapass/schema-service:v0.6.1-3 | Update to eclipse-pass/pass-metadata-schemas |
policyservice | oapass/policy-service:v0.1.2 | Update to eclipse-pass/pass-policy-service |
doiservice | oapass/doi-service:version1.0.0 | Update to eclipse-pass/pass-doi-service |
downloadservice | oapass/download-service:v1.0.2 | Update to eclipse-pass/pass-download-service |
The table below will document the progression from source code commits to a production running application for the many PASS components.
Project | Build | UTs | ITs | Deploy |
---|---|---|---|---|
pass-authz | ✅ | ✅ | ✅ | ❓ |
pass-data-model | ❌ | ❌ | ❌ | ❓ |
pass-ui | ✅ | ✅ | ❌ | ❓ |
pass-ui-public | ❌ | ❌ | ❌ | ❓ |
pass-ember-adapter | ✅ | ✅ | ❎ | ❓ |
pass-indexer | ✅ | ✅ | ✅ | ❓ |
pass-policy-service | ✅ | ✅ | ✅ | ❓ |
pass-metadata-schemas | ✅ | ✅ | ❌ | ❓ |
pass-download-service | ✅ | ❌ | ❎ | ❓ |
pass-notification-services | ✅ | ✅ | ✅ | ❓ |
pass-doi-service | ✅ | ✅ | ❎ | ❓ |
pass-deposit-services | ✅ | ✅ | ✅ | ❓ |
pass-indexer-checker | ✅ | ✅ | ✅ | ❓ |
pass-java-client | ✅ | ✅ | ✅ | ❓ |
pass-messaging-support | ✅ | ✅ | ✅ | ❓ |
pass-package-providers | ✅ | ✅ | ✅ | ❓ |
There are a few additional project that help support Eclipse PASS but are not part of the core application.
Project | Build | UTs | ITs | Deploy |
---|---|---|---|---|
pass-docker | ❌ | ❌ | ❌ | ❓ |
playground | ❌ | ❌ | ❌ | ❓ |
pass-dupe-checker | ✅ | ✅ | ❌ | ❓ |
pass-docker-mailserver | ❎ | ❌ | ❌ | ❓ |
The following application help load data into PASS
Project | Build | UTs | ITs | Deploy |
---|---|---|---|---|
pass-nihms-loader | ✅ | ✅ | ✅ | ❓ |
pass-grant-loader | ✅ | ✅ | ✅ | ❓ |
pass-journal-loader | ✅ | ✅ | ✅ | ❓ |
The following repositories have been forked from others
Project | Forked From |
---|---|
modeshape | ModeShape |
pass-fcrepo-module-auth-rbacl | birkland |
Project | Command | Dependencies |
---|---|---|
pass-ember-adapter | export FEDORA_ADAPTER_INTEGRATION_TEST=0 yarn install ember test |
Ember.js Ember |
pass-ui | npm config set spin false yarn install ember test –test-port=4200 |
NPM, Yarn, Ember.js |
pass-deposit-services | mvn test | Maven, Java |
pass-authz | mvn -pl -pass-authz-integration test | Maven, Java 8 |
pass-java-client | mvn test -DskipITs=true | Java, mvn test works if mvn package run first. |
pass-indexer | mvn test -DskipITs=true | Java |
pass-policy-service | go test ./… | Go |
pass-metadata-schemas | go test ./… | Go |
pass-download-service | go test ./… | Go |
pass-notification-services | mvn test | Java 8 |
pass-doi-service | mvn test | Java |
pass-indexer-checker | mvn test | Java |
pass-dupe-checker | go test | Go |
pass-package-providers | mvn test | Java, Mvn 3.6.3 (depending on platform, later release might work) |
pass-messaging-support | mvn test | Java |
pass-nihms-loader | mvn test | Java |
pass-grant-loader | mvn test | Java |
pass-journal-loader | mvn test | Java |
pass-fcrepo-module-auth-rbacl | mvn test | Java 8, Mvn 3.6.3 (depending on platform, later release might work) |
Project | Command | Dependencies |
---|---|---|
pass-ember-adapter | docker-compose up -d yarn install ember test |
Ember.js Ember Docker |
pass-deposit-services | mvn verify | Maven, Java, Docker |
pass-authz | mvn verify | Maven, Java 8, Docker |
pass-java-client | mvn verify | Java, Docker, mvn verify works if mvn package run first. |
pass-indexer | docker-compose up -d mvn verify |
Java, Docker |
pass-policy-service | docker-compose up -d go test -tags=integration ./… |
Go |
pass-download-service | docker-compose up -d go test -tags=integration ./… |
Go |
pass-notification-services | mvn verify | Java 8, Docker |
pass-doi-service | mvn verify | Java, Docker |
pass-indexer-checker | mvn verify | Java, Docker |
pass-package-providers | mvn verify | Java, Docker, Mvn 3.6.3 (depending on platform, later release might work) |
pass-messaging-support | mvn verify | Java, Docker |
pass-nihms-loader | mvn verify | Java, Docker |
pass-grant-loader | mvn verify | Java, Docker |
pass-journal-loader | mvn verify | Java, Docker |
pass-fcrepo-module-auth-rbacl | mvn test | Java 8, Mvn 3.6.3 (depending on platform, later release might work) |
Project | Command | Dependencies |
---|---|---|
pass-ember-adapter | yarn install ember build |
Ember.js Ember |
pass-ui | yarn install ember build |
Ember.js Ember |
pass-deposit-services | mvn install | Maven, Java, Docker |
pass-authz | mvn install | Maven, Java 8, Docker |
pass-java-client | mvn install | Java, Docker |
pass-indexer | docker-compose up -d mvn install |
Java, Docker |
pass-policy-service | go generate ./… go build ./cmd/pass-policy-service |
Go |
pass-docker-mailserver | docker build -t oapass/docker-mailserver:DATE . | Docker |
pass-metadata-schemas | go build ./cmd/pass-schema-service | Go |
pass-download-service | go build | Go |
pass-notification-services | mvn install | Java, Docker |
pass-doi-service | mvn install | Java, Docker |
pass-indexer-checker | mvn install | Java, Docker |
pass-dupe-checker | go build | Go |
pass-package-providers | mvn install | Java, Docker |
pass-messaging-support | mvn install | Java, Docker |
pass-nihms-loader | mvn install | Java, Docker |
pass-grant-loader | mvn install | Java, Docker |
pass-journal-loader | mvn install | Java, Docker |
Element | Current Version | Latest Version | Notes |
---|---|---|---|
Go | 1.12 | 1.18 | Programming language |
Java | 17 | 22 | Programming language |
Fedora | 4.7.5 | 6.1.1 | Document store (database) for PASS |
Postgres | 13.3 | 14.2 | Underlying storage of Fedora |
Node | 14.x | 18.x | JavaScript runtime, unconfirmed on desired version |
Npm | 6.14.x | 6.14.8 | Package manager for node |
Maven | 3.6.3 | 3.8.5 | Java Package manager |
ElasticSearch | 7.13.3 | 8.1.3 | Search / indexing |
Shibboleth | 2 | 3.3.0 | Authentication |
Ember.js | 3.8 | 3.28.8 | The user interface |
Ember CLI | 2.13 | 4.3.0 | CLI for managing an ember application |
Docker Compose | 1.29.2 | 2.5.0 | CLI for managing docker containers |