* Create a new pytest folder for live system testing with normal services (#15688)
* PoC for running dev env tests
* Replace in github actions
* Move folder to better location
* Further streamlining of new test folders
* Consolidate fixture, add writeup docs
* Use star import
* Push the wait-for-job to the conftest
Fix misused project cache identifier (#15690)
Fix project cache identifiers for new updates
Finish test and discover viable solution
Add comment on related task code
AAP-37989 Tests for exclude list with multiple jobs (#15722)
* Tests for exclude list with multiple jobs
Create test for using manual & file projects (#15754)
* Create test for using a manual project
* Chang default project factory to git, remove project files monkeypatch
* skip update of factory project
* Initial file scaffolding for feature
* Fill in galaxy and names
* Add README, describe project folders and dependencies
Add ee cleanup tests
* Adds cleanup tests to the live test.
Fix rsyslog permission error in github ubuntu tests from apparmor (#15717)
* Add test to detect rsyslog config problems
* Get dmesg output
* Disable apparmor for rsyslogd
Make awx/main/tests/live dramatically faster (#15780)
* Make awx/main/tests/live dramatically faster
* Add new setting to exclude list
* Fix rebase issues
* Did not want to backport this
* Publish image base on git repo name instead of hard coded to AWX (#15828)
* Fix git credential for devel_image build (#15834)
* Continue if pre-warm cache fail in container build (#15835)
* Use correct devel image for docker-compose (#15836)
PG_TLS=true make docker-compose
This will add some extra startup commands
for the postgres container to generate a key and
cert to use for postgres connections.
It will also mount in pgssl.conf which has ssl configuration.
This can be useful for debugging issues that only surface
when using ssl postgres connections.
* Update DOCKER_COMPOSE command
docker-compose will stop being supported soon and this is causing CI flake setting DOCKER_COMPOSE default to `docker compose`
* Give AWX network a static name
* Fixed mismatch between setuptools version in the makefile and requirements file
* Fix mismatch of versions in makefile and requirements
* Added maturin license
Not auto-reload explicitly STOPPED processes
In development/debug workflow sometime we explicitly STOP processes this will make sure auto-reload does not start them back up
`pytest awx/main/tests/docs --release=$(VERSION_TARGET)`
where --release is required breaks test discovery and running in vscode (from within the container)
* add ldap_auth mount and configure it
* added in key engines, userpass auth method, still needs testing
* add policies and fix ldap_user
* start awx automation for vault demo and move ldap
* update docs with new flags/new credentials
* Add TLS certificate auth for HashiCorp Vault
Add support for AWX to authenticate with HashiCorp Vault using
TLS client certificates.
Also updates the documentation for the HashiCorp Vault secret management
plugins to include both the new TLS options and the missing Kubernetes
auth method options.
Signed-off-by: Andrew Austin <aaustin@redhat.com>
* Refactor docker-compose vault for TLS cert auth
Add TLS configuration to the docker-compose Vault configuration and
use that method by default in vault plumbing.
This ensures that the result of bringing up the docker-compose stack
with vault enabled and running the plumb-vault playbook is a fully
working credential retrieval setup using TLS client cert authentication.
Signed-off-by: Andrew Austin <aaustin@redhat.com>
* Remove incorrect trailing space
Co-authored-by: Hao Liu <44379968+TheRealHaoLiu@users.noreply.github.com>
* Make vault init idempotent
- improve error handling for vault_initialization
- ignore error if vault cert auth is already configured
- removed unused register
* Add VAULT_TLS option
Make TLS for HashiCorp Vault optional and configurable via VAULT_TLS env var
* Add retries for vault init
Sometime it took longer for vault to fully come up and init will fail
---------
Signed-off-by: Andrew Austin <aaustin@redhat.com>
Co-authored-by: Hao Liu <44379968+TheRealHaoLiu@users.noreply.github.com>
Co-authored-by: Hao Liu <haoli@redhat.com>
* allow pytest --migrations to succeed
* We actually subvert migrations from running in test via pytest.ini
--no-migrations option. This has led to bit rot for the sqlite
migrations happy path. This changeset pays off that tech debt and
allows for an sqlite migration happy path.
* This paves the way for programatic invocation of individual migrations
and weaving of the creation of resources (i.e. Instance, Job Template,
etc). With this, a developer can instantiate various database states,
trigger a migration, assert the state of the db, and then have pytest
rollback all of that.
* I will note that in practice, running these migrations is dog shit
slow BUT this work also opens up the possibility of saving and
re-using sqlite3 database files. Normally, caching is not THE answer
and causes more harm than good. But in this case, our migrations are
mostly write-once (I say mostly because this change set violates
that :) so cache invalidation isn't a major issue.
* functional test for migrations on sqlite
* We commonly subvert running migrations in test land. Test land uses
sqlite. By not constantly exercising this code path it atrophies. The
smoke test here is to continuously exercise that code path.
* Add ci test to run migration tests separately, they take =~ 2-3
minutes each on my laptop.
* The smoke tests also serves as an example of how to write migration
tests.
* run migration tests in ci
There are a number of changes here:
- Abstract out a GHA composite action for running the dev environment
- Update the e2e tests to use that new abstracted action
- Introduce a new (matrixed) job for running collection integration
tests. This splits the jobs up based on filename.
- Collect coverage info and generate an html report that people can
download easily to see collection coverage info.
- Do some hacks to delete the intermediary coverage file artifacts
which aren't needed after the job finishes.
Signed-off-by: Rick Elrod <rick@elrod.me>
On some systems, /bin/sh is a bash symlink and running it will launch
bash in sh compatibility mode. However, bash-specific syntax will still
work in this mode (for example using == or pipefail).
However, on systems where /bin/sh is a symlink to another shell (think:
Debian-based) they might not have those bashisms.
Set the shell in the Makefile, so that it uses bash (since it is already
depending on bash, even though it is calling it as /bin/sh by default),
and add a shebang to pre-commit.sh for the same reason.
Signed-off-by: Rick Elrod <rick@elrod.me>
Added persistent storage
Auto-create vault and awx via playbooks
Create a new pattern for custom containers where we can do initialization
Auto-install roles needed for plumbing via the Makefile
- Change default PYTHON in Makefile to be ranked choice
- Fix `PYTHON_VERSION` target that expects just a word
- Use native GNU Make `$(subst ,,)` instead of `sed`
- Add 'version-for-buildyml' target to simplify ci
If I understand correctly, this change should make
'$(PYTHON)' work how we want it to everywhere. Before
this change, on develpers' machines that don't have
a 'python3.9' in their path, make would fail. With this
change, we will prefer python3.9 if it's available, but
we'll take python3 otherwise.
we link awx.egg-link from `tools/docker-compose/awx.egg-link` to `/tmp/awx.egg-link` than we move `/tmp/awx.egg-link` to `/var/lib/awx/venv/awx/lib/python3.9/site-packages/awx.egg-link`
bonus... now we dont have to set PYTHON=python3.9
- use different dockerfile for awx_devel and awx image
- make all Dockerfile* targets PHONY (bc its cheap to run)
- fix HEADLESS not working for awx-kube-build
these make targets are for starting the different daemons within the kube/docker development environment updating the name to make it better reflect their intention
also added comments above the make target to describe what they do
note: these comments show up when run `make help`
