changelog.md: Document v1.56.0

NC 32 and fixes

.github/ISSUE_TEMPLATE/1_Bug_report.md

@@ -5,7 +5,7 @@ about: Report errors and problems
 ---
 
 <!--
-Thanks for reporting issues back to NextCloudPi! 
+Thanks for reporting issues back to NextcloudPi! 
 
 Here you can file bugs and feature requests. **Do NOT ask questions**, this is not a support forum.
 
@@ -13,7 +13,7 @@ If there is an important security issue that has gone unnoticed, please send a p
 
 ### DO NOT ASK QUESTIONS, USE THE FORUMS
 
-NCP and its extras are explained in the wiki, also make sure to read the articles in ownyourbits explaining NextCloudPi extras before asking. https://ownyourbits.com/category/nextcloudpi/
+NCP and its extras are explained in the wiki, also make sure to read the articles in ownyourbits explaining NextcloudPi extras before asking. https://ownyourbits.com/category/nextcloudpi/
 
 For asking questions, please use the forums. https://help.nextcloud.com/c/support/appliances-docker-snappy-vm
 

.github/ISSUE_TEMPLATE/3_Support_question.md

@@ -4,7 +4,7 @@ about: Do NOT ask here - Use the forum
 
 ---
 
-Thanks for reporting issues back to NextCloudPi! 
+Thanks for reporting issues back to NextcloudPi! 
 
 Here you can file bugs and feature requests. **Do NOT ask questions**, this is not a support forum.
 
@@ -12,11 +12,11 @@ Here you can file bugs and feature requests. **Do NOT ask questions**, this is n
 DO NOT ASK QUESTIONS, USE THE FORUMS
 ---
 
-NCP and its extras are explained in the wiki, also make sure to read the articles in ownyourbits explaining NextCloudPi extras before asking. https://ownyourbits.com/category/nextcloudpi/
+NCP and its extras are explained in the wiki, also make sure to read the articles in ownyourbits explaining NextcloudPi extras before asking. https://ownyourbits.com/category/nextcloudpi/
 
-We use GitHub issues only to discuss about NextCloudPi bugs and new features.
+We use GitHub issues only to discuss about NextcloudPi bugs and new features.
 
-NCP and its extras are explained in the [wiki](https://github.com/nextcloud/nextcloudpi/wiki), also make sure to read the [articles in ownyourbits](https://ownyourbits.com/category/nextcloudpi/) explaining NextCloudPi extras before asking.
+NCP and its extras are explained in the [wiki](https://github.com/nextcloud/nextcloudpi/wiki), also make sure to read the [articles in ownyourbits](https://ownyourbits.com/category/nextcloudpi/) explaining NextcloudPi extras before asking.
 
 For any other question or comment, please use the [forums](https://help.nextcloud.com/c/support/appliances-docker-snappy-vm), and make sure to tag
 the thread as `ncp`.

.github/ISSUE_TEMPLATE/4_Documentation_issue.md

@@ -1,10 +1,10 @@
 ---
 name: ⛔ Documentation Issue
-about: See the NextCloudPi Wiki
+about: See the NextcloudPi Wiki
 
 ---
 
-The NextCloudPi wiki is open to anyone to edit. If you find something inaccurate,
+The NextcloudPi wiki is open to anyone to edit. If you find something inaccurate,
 missing or that can be improved, please feel free to contribute to it.
 
 https://github.com/nextcloud/nextcloudpi/wiki

.github/ISSUE_TEMPLATE/5_Security_issue.md

@@ -1,13 +0,0 @@
----
-name: ⛔ Security Issue
-about: Email nacho [at] ownyourbits.com
-
----
-
-⚠ PLEASE DON'T DISCLOSE SECURITY-RELATED ISSUES PUBLICLY, SEE BELOW.
-
-If you have found a security issue in NextCloudPi, please send the details to
-nacho [at] ownyourbits.com and don't disclose it publicly until we can provide a
-fix for it.
-
-Thank you!

.github/actions/create-test-instance-bullseye/action.yml

@@ -0,0 +1,35 @@
+name: Create Test VM
+description: Create NCP instance for testing in the Hetzner cloud
+inputs:
+  version:
+    description: version (git rev / tag / branch) to install
+    required: true
+  uid:
+    description: A unique ID for labeling/naming generated resources
+    required: true
+  hcloud_token:
+    description: A auth token for Hetzner cloud
+    required: true
+  server_type:
+    description: Server type to use for hetzner servers
+    required: true
+    default: "cx22"
+
+outputs:
+  server_address:
+    description: Adress of the test instance
+  snapshot_id:
+    description: ID of the generated postinstall snapshot
+  test_server_id:
+    description: ID of the created test server
+runs:
+  using: docker
+  image: docker://thecalcaholic/ncp-test-automation:bullseye
+
+  env:
+    HCLOUD_TOKEN: ${{ inputs.hcloud_token }}
+    UID: ${{ inputs.uid }}
+    SERVER_TYPE: ${{ inputs.server_type }}
+  args:
+    - /ncp-test-automation/bin/actions/create-test-instance.sh
+    - ${{ inputs.version }}

.github/actions/create-test-instance/action.yml

@@ -13,7 +13,7 @@ inputs:
   server_type:
     description: Server type to use for hetzner servers
     required: true
-    default: "cx11"
+    default: "cx22"
 
 outputs:
   server_address:
@@ -24,7 +24,7 @@ outputs:
     description: ID of the created test server
 runs:
   using: docker
-  image: docker://thecalcaholic/ncp-test-automation
+  image: docker://thecalcaholic/ncp-test-automation:latest
 
   env:
     HCLOUD_TOKEN: ${{ inputs.hcloud_token }}

.github/workflows/build-docker.yml

@@ -40,8 +40,8 @@ jobs:
       - name: Setup Docker Buildx
         uses: docker/setup-buildx-action@v2
 
-      - name: Set up QEMU
-        uses: docker/setup-qemu-action@v2
+#      - name: Set up QEMU
+#        uses: docker/setup-qemu-action@v3
 
       - name: Checkout code
         uses: actions/checkout@v3
@@ -119,8 +119,8 @@ jobs:
           echo "Previous version is '$version'"
           echo "PREVIOUS_VERSION=$version" >> "$GITHUB_ENV"
           echo "previous_version=${version}" >> $GITHUB_OUTPUT
-      - name: Set up QEMU
-        uses: docker/setup-qemu-action@v2
+#      - name: Set up QEMU
+#        uses: docker/setup-qemu-action@v3
 
       - name: Enable qemu SUID
         run: docker run --rm --privileged multiarch/qemu-user-static --reset -p yes --credential yes
@@ -381,7 +381,7 @@ jobs:
           docker exec nextcloudpi bash /usr/local/bin/ncp-diag |& awk "{ print \"${LOG_DIAG} \" \$0 }"
       - name: "Upload error report"
         if: ${{ failure() }}
-        uses: actions/upload-artifact@v3
+        uses: actions/upload-artifact@v4
         with:
           name: ${{ github.run_id }}-docker-${{ env.ARCH }}-update-test-error-report
           path: tests/error-report.txt
@@ -399,8 +399,8 @@ jobs:
       LOG_TEST: "\\033[1;33mTEST::\\033[0m"
       LOG_DIAG: "\\033[1;31mDIAG::\\033[0m"
     steps:
-      - name: Set up QEMU
-        uses: docker/setup-qemu-action@v2
+#      - name: Set up QEMU
+#        uses: docker/setup-qemu-action@v3
 
       - name: Login to docker
         run: |
@@ -533,7 +533,7 @@ jobs:
 
       - name: "Upload error report"
         if: ${{ failure() }}
-        uses: actions/upload-artifact@v3
+        uses: actions/upload-artifact@v4
         with:
           name: ${{ github.run_id }}-docker-${{ env.ARCH }}-install-test-error-report
           path: tests/error-report.txt

.github/workflows/build-lxd.yml

@@ -31,26 +31,31 @@ jobs:
   determine-runner:
     env:
       LXD_ARCH: "${{ inputs.arch || 'x86' }}"
-    runs-on: ubuntu-20.04
+    runs-on: ubuntu-latest
     outputs:
       runner_label: ${{ steps.script.outputs.runner_label }}
+      lxc_cmd: ${{ steps.script.outputs.lxc_cmd }}
+      lxd_extra_profile: ${{ steps.script.outputs.lxd_extra_profile }}
     steps:
       - name: script
         id: script
         run: |
           if [[ "${LXD_ARCH}" == "arm64" ]]
           then
-            RUNNER_LABEL=ubuntu-20.04-arm64
+            LXC_CMD="incus"
+            RUNNER_LABEL="ubuntu-20.04-arm64"
+            echo "lxd_extra_profile=network" | tee -a "$GITHUB_OUTPUT"
           else
-            RUNNER_LABEL=ubuntu-20.04
+            LXC_CMD="incus"
+            RUNNER_LABEL="ubuntu-latest"
           fi
-          echo "runner_label=$RUNNER_LABEL" | tee -a $GITHUB_OUTPUT
 
+          echo "runner_label=$RUNNER_LABEL" | tee -a $GITHUB_OUTPUT
+          echo "lxc_cmd=$LXC_CMD" | tee -a $GITHUB_OUTPUT
   build-current:
     needs:
       - determine-runner
-    runs-on:
-      - ${{ needs.determine-runner.outputs.runner_label }}
+    runs-on: ${{ needs.determine-runner.outputs.runner_label }}
     outputs:
       artifact_name: "${{ env.ARTIFACT_NAME }}"
       artifact_file: "${{ steps.pack-lxd.outputs.artifact_file }}"
@@ -58,239 +63,558 @@ jobs:
       VERSION: "${{ inputs.git_ref || github.head_ref || github.ref_name }}"
       ARTIFACT_NAME: "${{ github.run_id }}-lxd-${{ inputs.arch || 'x86' }}-image"
       LXD_ARCH: "${{ inputs.arch || 'x86' }}"
+      LXC: "${{ needs.determine-runner.outputs.lxc_cmd }}"
+      LXD_EXTRA_PROFILE: "${{ needs.determine-runner.outputs.lxd_extra_profile }}"
     steps:
       - name: Checkout code
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
         with:
           ref: "${{ env.VERSION }}"
-      - name: Cleanup lxd
-        run: test -z "$(lxc profile device show default | grep eth0)" || lxc profile device remove default eth0
-      - uses: whywaita/setup-lxd@v1
-        with:
-          lxd_version: latest/stable
-      - name: Build LXD image
+      - name: Setup incus
+        if: ${{ needs.determine-runner.outputs.runner_label != 'ubuntu-20.04-arm64' }}
         run: |
-          ./build/build-LXD.sh
+          sudo apt-get install -y incus qemu-system incus-tools
+          sudo iptables -I DOCKER-USER -i incusbr0 -j ACCEPT
+          sudo iptables -I DOCKER-USER -o incusbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
+          sudo incus admin init --auto
+      - name: Build LXD image
+        env:
+          USE_INCUS: "${{ needs.determine-runner.outputs.lxc_cmd == 'incus' && 'yes' || 'no' }}"
+        run: |
+          export CI
+          LXD_EXTRA_PROFILE="$LXD_EXTRA_PROFILE" BRANCH="${VERSION##refs/heads/}" ./build/build-LXD.sh
       - name: Pack LXD image
         id: pack-lxd
         run: |
           . ./build/buildlib.sh
-          ARTIFACT_FILE="NextCloudPi_LXD_${LXD_ARCH:-x86}_${VERSION//\//_}"
-          lxc image export -q ncp/"${version}" "output/${ARTIFACT_FILE}"
+          ARTIFACT_FILE="NextcloudPi_LXD_${LXD_ARCH:-x86}_${VERSION//\//_}"
+          sudo "$LXC" image export -q ncp/"${version}" "output/${ARTIFACT_FILE}"
           echo "artifact_file=${ARTIFACT_FILE}.tar.gz" >> $GITHUB_OUTPUT
       - name: upload LXD image to artifact store
-        uses: actions/upload-artifact@v3
+        uses: actions/upload-artifact@v4
         with:
           name: "${{ env.ARTIFACT_NAME }}"
           path: "output/${{ steps.pack-lxd.outputs.artifact_file }}"
           if-no-files-found: error
 
-  build-previous:
-    runs-on: ubuntu-20.04
-    if: ${{ inputs.arch == 'x86' || inputs.arch == '' }}
-    outputs:
-      artifact_name: "${{ env.ARTIFACT_NAME }}"
-      artifact_file: "${{ steps.pack-lxd.outputs.artifact_file }}"
-      previous_version: "${{ steps.checkout_previous_version.outputs.previous_version }}"
-    env:
-      VERSION: "${{ inputs.git_ref || github.head_ref || github.ref_name }}"
-      ARTIFACT_NAME: "${{ github.run_id }}-lxd-${{ inputs.arch || 'x86' }}-image-previous"
-      LXD_EXTRA_PROFILE: network
-      LXD_ARCH: "${{ inputs.arch || 'x86' }}"
-    steps:
-      - name: Checkout code
-        uses: actions/checkout@v3
-        with:
-          ref: "${{ env.VERSION }}"
-          fetch-depth: 0
-      - name: Checkout previous release
-        id: checkout_previous_version
-        shell: bash
-        run: |
-          set -e
-          if [[ -n "${{ github.base_ref }}" ]]
-          then
-            version="${{ github.base_ref }}"
-          elif [[ "${{ env.VERSION }}" == "refs/heads/devel" ]]
-          then
-            version="master"
-          else
-            if [[ "${{ github.ref_type }}" != "tag" ]] || ! git describe --tags > /dev/null
-            then
-              git fetch -fu --tags origin ${{ env.VERSION }}:${{ env.VERSION }}
-            fi
-            version="$(git describe --tags)"
-            [[ "$version" =~ .*-.*-.* ]] || {
-              git checkout HEAD~1
-              version="$(git describe --tags)"
-            }
-            version="${version%-*-*}"
-          fi
-          echo "Previous version is '$version'"
-          git checkout "$version"
-          echo "VERSION=$version" >> "$GITHUB_ENV"
-          echo "previous_version=${version}" >> $GITHUB_OUTPUT
-      - uses: whywaita/setup-lxd@v1
-        with:
-          lxd_version: latest/stable
-      - name: Fix LXD
-        run: |
-          lxc profile create network
-          cat <<EOF | lxc profile edit network
-          devices:
-            eth0:
-              name: eth0
-              parent: lxdbr0
-              nictype: bridged
-              type: nic
-          EOF
+#  build-previous:
+#    needs:
+#      - determine-runner
+#    runs-on: [ ubuntu-20.04 ]
+#    if: ${{ inputs.arch == 'x86' || inputs.arch == '' }}
+#    outputs:
+#      artifact_name: "${{ env.ARTIFACT_NAME }}"
+#      artifact_file: "${{ steps.pack-lxd.outputs.artifact_file }}"
+#      previous_version: "${{ steps.checkout_previous_version.outputs.previous_version }}"
+#    env:
+#      VERSION: "${{ inputs.git_ref || github.head_ref || github.ref_name }}"
+#      ARTIFACT_NAME: "${{ github.run_id }}-lxd-${{ inputs.arch || 'x86' }}-image-previous"
+#      LXD_ARCH: "${{ inputs.arch || 'x86' }}"
+#      LXC: "${{ needs.determine-runner.outputs.lxc_cmd }}"
+#      LXD_EXTRA_PROFILE: "${{ needs.determine-runner.outputs.lxd_extra_profile }}"
+#    steps:
+#      - name: Checkout code
+#        uses: actions/checkout@v3
+#        with:
+#          ref: "${{ env.VERSION }}"
+#          fetch-depth: 0
+#      - name: Checkout previous release
+#        id: checkout_previous_version
+#        shell: bash
+#        run: |
+#          set -e
+#          if [[ -n "${{ github.base_ref }}" ]]
+#          then
+#            version="${{ github.base_ref }}"
+#          elif [[ "${{ env.VERSION }}" == "refs/heads/devel" ]]
+#          then
+#            version="master"
+#          else
+#            if [[ "${{ github.ref_type }}" != "tag" ]] || ! git describe --tags > /dev/null
+#            then
+#              git fetch -fu --tags origin ${{ env.VERSION }}:${{ env.VERSION }}
+#            fi
+#            version="$(git describe --tags)"
+#            [[ "$version" =~ .*-.*-.* ]] || {
+#              git checkout HEAD~1
+#              version="$(git describe --tags)"
+#            }
+#            version="${version%-*-*}"
+#          fi
+#          echo "Previous version is '$version'"
+#          git checkout "$version"
+#          echo "VERSION=$version" >> "$GITHUB_ENV"
+#          echo "previous_version=${version}" >> $GITHUB_OUTPUT
+#      - name: Setup incus
+#        if: ${{ needs.determine-runner.outputs.runner_label != 'ubuntu-20.04-arm64' }}
+#        run: |
+#          curl https://pkgs.zabbly.com/get/incus-stable | sudo sh -x
+#          sudo iptables -I DOCKER-USER -i incusbr0 -j ACCEPT
+#          sudo iptables -I DOCKER-USER -o incusbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
+#          sudo incus admin init --auto
+#
+#      - name: Build LXD image
+#        env:
+#          USE_INCUS: "${{ needs.determine-runner.outputs.lxc_cmd == 'incus' && 'yes' || 'no' }}"
+#        run: |
+#          export CI
+#          LXD_EXTRA_PROFILE="$LXD_EXTRA_PROFILE" BRANCH="${VERSION##refs/heads/}" ./build/build-LXD.sh
+#      - name: Pack LXD image
+#        id: pack-lxd
+#        run: |
+#          . ./build/buildlib.sh
+#          ARTIFACT_FILE="NextcloudPi_LXD_${LXD_ARCH:-x86}_${VERSION//\//_}"
+#          sudo "${LXC}" image export -q ncp/"${version}" "output/${ARTIFACT_FILE}"
+#          echo "artifact_file=${ARTIFACT_FILE}.tar.gz" >> $GITHUB_OUTPUT
+#      - name: upload LXD image to artifact store
+#        uses: actions/upload-artifact@v3
+#        with:
+#          name: "${{ env.ARTIFACT_NAME }}"
+#          path: "output/${{ steps.pack-lxd.outputs.artifact_file }}"
+#          if-no-files-found: error
+#
+#  test-update:
+#    needs:
+#      - determine-runner
+#      - build-previous
+#    runs-on: ${{ needs.determine-runner.outputs.runner_label }}
+#    outputs:
+#      artifact_name: "${{ env.ARTIFACT_NAME }}"
+#      artifact_file: "${{ steps.pack-lxd.outputs.artifact_file }}"
+#    env:
+#      VERSION: "${{ inputs.git_ref || github.head_ref || github.ref_name }}"
+#      ARTIFACT_NAME: "${{ needs.build-previous.outputs.artifact_name }}"
+#      LXC: "${{ needs.determine-runner.outputs.lxc_cmd }}"
+#      USE_INCUS: "${{ needs.determine-runner.outputs.lxc_cmd == 'incus' && 'yes' || 'no' }}"
+#    steps:
+#      - name: Checkout code
+#        uses: actions/checkout@v3
+#        with:
+#          ref: "${{ env.VERSION }}"
+#      - name: Setup incus
+#        if: ${{ needs.determine-runner.outputs.runner_label != 'ubuntu-20.04-arm64' }}
+#        run: |
+#          curl https://pkgs.zabbly.com/get/incus-stable | sudo sh -x
+#          sudo iptables -I DOCKER-USER -i incusbr0 -j ACCEPT
+#          sudo iptables -I DOCKER-USER -o incusbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
+#          sudo incus admin init --auto
+#      - name: Setup Firefox
+#        continue-on-error: true
+#        id: setup-firefox-browser-action
+#        uses: browser-actions/setup-firefox@latest
+#      - name: Setup Firefox from packages
+#        if: ${{ steps.setup-firefox-browser-action.outcome == 'failure' }}
+#        run: |
+#          sudo apt-get install -y --no-install-recommends firefox
+#      - name: Setup GeckoDriver
+#        env:
+#          GH_TOKEN: ${{ github.token }}
+#        run: |
+#          arch=linux64
+#          if [[ "$RUNNER_ARCH" == "ARM64" ]]
+#          then
+#            arch="linux-aarch64"
+#          fi
+#          gh release download -R mozilla/geckodriver -p 'geckodriver-*-'"$arch.tar.gz"
+#          tar xf "geckodriver-"*"-$arch.tar.gz"
+#          sudo mv geckodriver /usr/local/bin/
+#      - name: Setup Selenium
+#        run: pip install selenium
+#      - name: download LXD image from artifact store
+#        uses: actions/download-artifact@v3
+#        with:
+#          name: ${{ needs.build-previous.outputs.artifact_name }}
+#      - name: Remove old lxd images
+#        if: ${{ inputs.arch == 'arm64' }}
+#        run: |
+#          for img in $("$LXC" image list -c f -f csv)
+#          do
+#            sudo "$LXC" image delete "$img"
+#          done
+#      - name: Launch ncp container
+#        run: |
+#          set -x
+#          sudo "$LXC" delete -q -f ncp || true
+#          sudo "$LXC" image import -q "./${{ needs.build-previous.outputs.artifact_file }}" --alias "ncp/update"
+#          systemd-run --user --scope -p "Delegate=yes" "$LXC" launch -q "ncp/test" ncp || \
+#          sudo systemd-run --scope -p "Delegate=yes" "$LXC" launch -q "ncp/test" ncp
+#          sudo "$LXC" exec ncp -- bash -c 'while [ "$(systemctl is-system-running 2>/dev/null)" != "running" ] && [ "$(systemctl is-system-running 2>/dev/null)" != "degraded" ]; do :; done'
+#          sudo "$LXC" exec ncp -- rm -f /opt/ncdata/data/nextcloud.log
+#          sleep 30
+#          ip="$(sudo "$LXC" list -c n4 -f csv | grep '^ncp' | cut -d ',' -f2)"
+#          ip="${ip/% *}"
+#          sudo sed -i 's/^.*nextcloudpi.local.*$//g' /etc/hosts
+#          echo "${ip} nextcloudpi.local" | sudo tee -a /etc/hosts
+#      - name: Activate and Test LXD Image
+#        working-directory: ./tests
+#        run: |
+#          sudo "$LXC" exec ncp -- bash -c 'tail -f /var/log/ncp.log' |& awk '{ print "NCP::" $0 }' &
+#          python activation_tests.py --no-gui "nextcloudpi.local" 443 4443 || {
+#            echo "Activation test failed!"
+#            echo "Geckodriver logs:"
+#            tail -n 20 geckodriver.log >&2 || true
+#            echo "================"
+#            echo "ncp.log: "
+#            "$LXC" exec ncp -- "tail -n20 /var/log/ncp.log" || true
+#            exit 1
+#          }
+#          python nextcloud_tests.py --no-gui --skip-release-check "nextcloudpi.local" 443 4443 || {
+#            echo "Nextcloud test failed!"
+#            echo "Geckodriver logs:"
+#            tail -n 20 geckodriver.log >&2 || true
+#            echo "================"
+#            echo "ncp.log: "
+#            "$LXC" exec ncp -- "tail -n20 /var/log/ncp.log" || true
+#            echo "================"
+#            echo "nextcloud log: "
+#            datadir="$("$LXC" exec ncp -- ncc config:system:get datadirectory)"
+#            "$LXC" exec ncp -- cat "$datadir/nextcloud.log" || true
+#            exit 1
+#          }
+#          USE_INCUS="$USE_INCUS" python system_tests.py --non-interactive --skip-update-test || {
+#            echo "System test failed!"
+#            exit 1
+#          }
+#      - name: Update ncp
+#        env:
+#          HEAD_REF: "${{ github.head_ref }}"
+#        run: |
+#          set -ex
+#          BRANCH="${VERSION/refs\/heads\//}"
+#          BRANCH="${BRANCH/refs\/tags\//}"
+#          if [[ "$BRANCH" =~ "refs/pull/"* ]]
+#          then
+#            UPDATE_ARGS=("${HEAD_REF}" "$VERSION")
+#          else
+#            UPDATE_ARGS=("$BRANCH")
+#          fi
+#          echo "VERSION=${VERSION}" >> "$GITHUB_ENV"
+#          echo "Running update ${{ needs.build-previous.outputs.previous_version }} -> ${VERSION}"
+#
+#          current_nc_version="$(sudo "$LXC" exec ncp -- ncc status | grep "version:" | awk '{ print $3 }')"
+#          latest_nc_version="$(cat etc/ncp.cfg | jq -r '.nextcloud_version')"
+#
+#          sudo "$LXC" exec ncp -- bash -c "DBG=x ncp-update ${UPDATE_ARGS[*]}"
+#          sudo "$LXC" exec ncp -- /usr/local/bin/ncc status
+#
+#          if [[ "$current_nc_version" =~ "$latest_nc_version".* ]]
+#          then
+#            echo "Nextcloud is up to date - skipping NC update test."
+#          else
+#            sudo "$LXC" exec ncp -- bash -c "DBG=x ncp-update-nc ${latest_nc_version?}"
+#          fi
+#
+#          sudo "$LXC" exec ncp -- rm -f /opt/ncdata/data/nextcloud.log
+#
+#          sudo "$LXC" stop ncp
+#      - name: Relaunch container
+#        run: |
+#          set -x
+#          systemd-run --user --scope -p "Delegate=yes" "$LXC" start -q ncp || \
+#          sudo systemd-run --scope -p "Delegate=yes" "$LXC" start -q ncp
+#          sudo "$LXC" exec ncp -- bash -c 'while [ "$(systemctl is-system-running 2>/dev/null)" != "running" ] && [ "$(systemctl is-system-running 2>/dev/null)" != "degraded" ]; do :; done'
+#          sleep 30
+#          ip="$(sudo "$LXC" list -c n4 -f csv | grep '^ncp' | cut -d ',' -f2)"
+#          ip="${ip/% *}"
+#          sudo sed -i 's/^.*nextcloudpi.local.*$//g' /etc/hosts
+#          echo "${ip} nextcloudpi.local" | sudo tee -a /etc/hosts
+#      - name: Test LXD Image
+#        working-directory: ./tests
+#        run: |
+#          sudo "$LXC" exec ncp -- bash -c 'tail -f /var/log/ncp.log' |& awk '{ print "NCP::" $0 }' &
+#          python nextcloud_tests.py --no-gui --skip-release-check "nextcloudpi.local" 443 4443 || {
+#            echo "Nextcloud test failed!"
+#            echo "Geckodriver logs:"
+#            tail -n 20 geckodriver.log >&2 || true
+#            echo "================"
+#            echo "ncp.log: "
+#            sudo "$LXC" exec ncp -- "tail -n20 /var/log/ncp.log" || true
+#            echo "================"
+#            echo "nextcloud log: "
+#            datadir="$("$LXC" exec ncp -- ncc config:system:get datadirectory)"
+#            sudo "$LXC" exec ncp -- cat "$datadir/nextcloud.log" || true
+#            exit 1
+#          }
+#          USE_INCUS="$USE_INCUS" python system_tests.py --non-interactive || {
+#            echo "System test failed!"
+#            exit 1
+#          }
 
-      - name: Set up QEMU
-        uses: docker/setup-qemu-action@v2
-      - name: Enable qemu SUID
-        run: docker run --rm --privileged multiarch/qemu-user-static --reset -p yes --credential yes
-      - name: Download qemu binaries
-        run: |
-          wget -nv https://github.com/multiarch/qemu-user-static/releases/latest/download/qemu-aarch64-static -O qemu-aarch64-static
-          wget -nv https://github.com/multiarch/qemu-user-static/releases/latest/download/qemu-arm-static -O qemu-arm-static
-          chmod +x qemu-{arm,aarch64}-static
-
-      - name: Build LXD image
-        run: |
-          ./build/build-LXD.sh
-      - name: Pack LXD image
-        id: pack-lxd
-        run: |
-          . ./build/buildlib.sh
-          ARTIFACT_FILE="NextCloudPi_LXD_${LXD_ARCH:-x86}_${VERSION//\//_}"
-          lxc image export -q ncp/"${version}" "output/${ARTIFACT_FILE}"
-          echo "artifact_file=${ARTIFACT_FILE}.tar.gz" >> $GITHUB_OUTPUT
-      - name: upload LXD image to artifact store
-        uses: actions/upload-artifact@v3
-        with:
-          name: "${{ env.ARTIFACT_NAME }}"
-          path: "output/${{ steps.pack-lxd.outputs.artifact_file }}"
-          if-no-files-found: error
-
-  test-update:
+  test-dist-upgrade:
+    if: false
     needs:
-      - build-previous
-    runs-on: ubuntu-20.04
-    outputs:
-      artifact_name: "${{ env.ARTIFACT_NAME }}"
-      artifact_file: "${{ steps.pack-lxd.outputs.artifact_file }}"
+      - determine-runner
+    runs-on: ${{ needs.determine-runner.outputs.runner_label }}
     env:
       VERSION: "${{ inputs.git_ref || github.ref }}"
-      ARTIFACT_NAME: "${{ needs.build-previous.outputs.artifact_name }}"
+      LXC: "${{ needs.determine-runner.outputs.lxc_cmd }}"
+      LXD_EXTRA_PROFILE: "${{ needs.determine-runner.outputs.lxd_extra_profile }}"
+      PREVIOUS_IMAGE_URL_ARM64: "https://github.com/nextcloud/nextcloudpi/releases/download/v1.53.2/NextcloudPi_LXD_arm64_v1.53.2.tar.gz"
+      PREVIOUS_IMAGE_URL_AMD64: "https://github.com/nextcloud/nextcloudpi/releases/download/v1.53.2/NextCloudPi_LXD_x86_v1.53.2.tar.gz"
+      USE_INCUS: "${{ needs.determine-runner.outputs.lxc_cmd == 'incus' && 'yes' || 'no' }}"
     steps:
-      - uses: whywaita/setup-lxd@v1
-        with:
-          lxd_version: latest/stable
-      - name: Fix LXD
+      - name: Setup incus
+        if: ${{ needs.determine-runner.outputs.runner_label != 'ubuntu-20.04-arm64' }}
         run: |
-          lxc profile create network
-          cat <<EOF | lxc profile edit network
-          devices:
-            eth0:
-              name: eth0
-              parent: lxdbr0
-              nictype: bridged
-              type: nic
-          EOF
-      - name: Setup Firefox
-        uses: browser-actions/setup-firefox@latest
-      - name: Setup GeckoDriver
-        uses: ChlodAlejandro/setup-geckodriver@latest
-        with:
-          token: ${{ secrets.GITHUB_TOKEN }}
-      - name: Setup Selenium
-        run: pip install selenium
+          sudo apt-get install -y incus qemu-system incus-tools
+          sudo iptables -I DOCKER-USER -i incusbr0 -j ACCEPT
+          sudo iptables -I DOCKER-USER -o incusbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
+          sudo incus admin init --auto
       - name: Checkout code
         uses: actions/checkout@v3
         with:
-          ref: "${{ env.VERSION }}"
-      - name: download LXD image from artifact store
-        uses: actions/download-artifact@v3
-        with:
-          name: ${{ needs.build-previous.outputs.artifact_name }}
+          ref: "v1.54.3"
+      - name: Setup Firefox
+        if: ${{ runner.arch != 'ARM64' }}
+        continue-on-error: true
+        id: setup-firefox-browser-action
+        uses: browser-actions/setup-firefox@latest
+      - name: Setup Firefox from packages
+        if: ${{ runner.arch == 'ARM64' || steps.setup-firefox-browser-action.outcome == 'failure' }}
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y --no-install-recommends firefox
+      - name: Setup GeckoDriver
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: |
+          set -x
+          arch=linux64
+          if [[ "${{ runner.arch }}" == "ARM64" ]]
+          then
+            arch="linux-aarch64"
+          fi
+          gh release download -R mozilla/geckodriver -p 'geckodriver-*-'"$arch.tar.gz"
+          tar xf "geckodriver-"*"-$arch.tar.gz"
+          sudo mv geckodriver /usr/local/bin/
+          sudo chmod +x /usr/local/bin/geckodriver
+          echo "GECKODRIVER_PATH=/usr/local/bin/geckodriver" >> "$GITHUB_ENV"
+          geckodriver -V
+      - name: Setup Selenium
+        run: |
+          sudo apt-get -y install python3-venv
+          python3 -m venv ./.venv
+          ./.venv/bin/pip install selenium
+      - name: download LXD image
+        run: |
+          image_url="${PREVIOUS_IMAGE_URL_AMD64?}"
+          if [[ "${{ runner.arch }}" == "ARM64" ]]
+          then
+            image_url="${PREVIOUS_IMAGE_URL_ARM64?}"
+          fi
+          wget -qO ./ncp.tar.gz "${image_url?}"
       - name: Launch ncp container
         run: |
           set -x
-          lxc delete -q -f ncp || true
-          lxc image import -q "./${{ needs.build-previous.outputs.artifact_file }}" --alias "ncp/update"
-          systemd-run --user --scope -p "Delegate=yes" lxc launch -q "ncp/update" ncp
-          lxc exec ncp -- bash -c 'while [ "$(systemctl is-system-running 2>/dev/null)" != "running" ] && [ "$(systemctl is-system-running 2>/dev/null)" != "degraded" ]; do :; done'
+          sudo "$LXC" delete -q -f ncp || true
+          sudo "$LXC" image import -q "./ncp.tar.gz" --alias "ncp/update"
+          LXC_ARGS=(-p default)
+          [[ -z "$LXD_EXTRA_PROFILE" ]] || LXC_ARGS+=(-p "$LXD_EXTRA_PROFILE")
+          systemd-run --user --scope -p "Delegate=yes" "$LXC" launch -q "${LXC_ARGS[@]}" "ncp/update" ncp || \
+          sudo systemd-run --scope -p "Delegate=yes" "$LXC" launch -q "${LXC_ARGS[@]}" "ncp/update" ncp
+          sudo "$LXC" exec ncp -- bash -c 'while [ "$(systemctl is-system-running 2>/dev/null)" != "running" ] && [ "$(systemctl is-system-running 2>/dev/null)" != "degraded" ]; do :; done'
+          sudo "$LXC" exec ncp -- rm -f /opt/ncdata/data/nextcloud.log
           sleep 30
-          ip="$(lxc list -c n4 -f csv | grep '^ncp' | cut -d ',' -f2)"
+          ip="$(sudo "$LXC" list -c n4 -f csv | grep '^ncp' | cut -d ',' -f2)"
           ip="${ip/% *}"
-          echo "${ip} nextcloudpi.local" | sudo tee /etc/hosts
+          sudo sed -i 's/^.*nextcloudpi.local.*$//g' /etc/hosts
+          echo "${ip} nextcloudpi.local" | sudo tee -a /etc/hosts
+      - name: Activate and Test LXD Image
+        working-directory: ./tests
+        run: |
+          set -x
+          export GECKODRIVER_PATH="$GECKODRIVER_PATH"
+          export FF_BINARY_PATH="$(which firefox)"
+          sudo "$LXC" exec ncp -- bash -c 'tail -f /var/log/ncp.log' |& awk '{ print "NCP::" $0 }' &
+          ../.venv/bin/python activation_tests.py --no-gui "nextcloudpi.local" 443 4443 || {
+            echo "Activation test failed!"
+            echo "Geckodriver logs:"
+            tail -n 20 geckodriver.log >&2 || true
+            echo "================"
+            echo "ncp.log: "
+            sudo "$LXC" exec ncp -- "tail -n20 /var/log/ncp.log" || true
+            exit 1
+          }
+          ../.venv/bin/python nextcloud_tests.py --skip-release-check --no-gui "nextcloudpi.local" 443 4443 || {
+            echo "Nextcloud test failed!"
+            echo "Geckodriver logs:"
+            tail -n 20 geckodriver.log >&2 || true
+            echo "================"
+            echo "ncp.log: "
+            sudo "$LXC" exec ncp -- "tail -n20 /var/log/ncp.log" || true
+            echo "================"
+            echo "nextcloud log: "
+            datadir="$(sudo "$LXC" exec ncp -- ncc config:system:get datadirectory)"
+            sudo "$LXC" exec ncp -- cat "$datadir/nextcloud.log" || true
+            exit 1
+          }
+          USE_INCUS="$USE_INCUS"  ../.venv/bin/python system_tests.py --non-interactive --skip-update-test || {
+            echo "System test failed!"
+            echo "ncp.log: "
+            sudo "$LXC" exec ncp -- "tail -n20 /var/log/ncp.log" || true
+            echo "================"
+            echo "nextcloud log: "
+            datadir="$(sudo "$LXC" exec ncp -- ncc config:system:get datadirectory)"
+            sudo "$LXC" exec ncp -- cat "$datadir/nextcloud.log" || true
+            exit 1
+          }
 
       - name: Update ncp
+        env:
+          HEAD_REF: "${{ github.head_ref }}"
         run: |
           set -ex
           BRANCH="${VERSION/refs\/heads\//}"
           BRANCH="${BRANCH/refs\/tags\//}"
           if [[ "$BRANCH" =~ "refs/pull/"* ]]
           then
-            UPDATE_ARGS=("${{ github.base_ref }}" "$VERSION") 
+            UPDATE_ARGS=("${HEAD_REF}" "$VERSION") 
           else
             UPDATE_ARGS=("$BRANCH")
           fi
           echo "VERSION=${VERSION}" >> "$GITHUB_ENV"
-          echo "Running update ${{ needs.build-previous.outputs.previous_version }} -> ${VERSION}"
+          echo "Running update to ${VERSION}"
           
-          current_nc_version="$(lxc exec ncp -- ncc status | grep "version:" | awk '{ print $3 }')"
-          latest_nc_version="$(cat etc/ncp.cfg | jq -r '.nextcloud_version')"
+          current_nc_version="$(sudo "$LXC" exec ncp -- ncc status | grep "version:" | awk '{ print $3 }')"
+          latest_nc_version="29.0.9"
           
-          lxc exec ncp -- bash -c "DBG=x ncp-update ${UPDATE_ARGS[*]}"
-          lxc exec ncp -- /usr/local/bin/ncc status
+          sudo "$LXC" exec ncp -- apt-get update ||:
+          sudo "$LXC" exec ncp -- apt-get install --no-install-recommends -y gnupg2
+          sudo "$LXC" exec ncp -- apt-key adv --fetch-keys https://packages.sury.org/php/apt.gpg
           
-          if [[ "$current_nc_version" =~ "$latest_nc_version".* ]]
-          then
-            echo "Nextcloud is up to date - skipping NC update test."
-          else
-            lxc exec ncp -- bash -c "DBG=x ncp-update-nc ${latest_nc_version?}"
-          fi
+          sudo "$LXC" exec ncp -- bash -c "DBG=x ncp-update ${UPDATE_ARGS[*]}"
+          sudo "$LXC" exec ncp -- /usr/local/bin/ncc status
           
-          lxc stop ncp
-          lxc publish -q ncp -f --alias "ncp/updated"
+          #if [[ "$current_nc_version" =~ "$latest_nc_version".* ]]
+          #then
+          #  echo "Nextcloud is up to date - skipping NC update test."
+          #else
+          #  sudo "$LXC" exec ncp -- bash -c "DBG=x ncp-update-nc ${latest_nc_version?}"
+          #  sudo "$LXC" exec ncp -- /usr/local/bin/ncc status
+          #fi
+  
+          sudo "$LXC" exec ncp -- rm -f /opt/ncdata/data/nextcloud.log
+          
+          sudo "$LXC" stop ncp
       - name: Relaunch container
         run: |
           set -x
-          lxc delete -q -f ncp || true
-          systemd-run --user --scope -p "Delegate=yes" lxc launch -q "ncp/updated" ncp
-          lxc exec ncp -- bash -c 'while [ "$(systemctl is-system-running 2>/dev/null)" != "running" ] && [ "$(systemctl is-system-running 2>/dev/null)" != "degraded" ]; do :; done'
+          LXC_ARGS=()
+          [[ -z "$LXD_EXTRA_PROFILE" ]] || LXC_ARGS+=(-p "$LXD_EXTRA_PROFILE")
+          systemd-run --user --scope -p "Delegate=yes" "$LXC" start ncp || \
+          sudo systemd-run --scope -p "Delegate=yes" "$LXC" start ncp
+          sudo "$LXC" exec ncp -- bash -c 'while [ "$(systemctl is-system-running 2>/dev/null)" != "running" ] && [ "$(systemctl is-system-running 2>/dev/null)" != "degraded" ]; do :; done'
           sleep 30
-          ip="$(lxc list -c n4 -f csv | grep '^ncp' | cut -d ',' -f2)"
+          ip="$(sudo "$LXC" list -c n4 -f csv | grep '^ncp' | cut -d ',' -f2)"
           ip="${ip/% *}"
-          echo "${ip} nextcloudpi.local" | sudo tee /etc/hosts
+          sudo sed -i 's/^.*nextcloudpi.local.*$//g' /etc/hosts
+          echo "${ip} nextcloudpi.local" | sudo tee -a /etc/hosts
       - name: Test LXD Image
         working-directory: ./tests
         run: |
-          lxc exec ncp -- bash -c 'tail -f /var/log/ncp.log' |& awk '{ print "NCP::" $0 }' &
-          python activation_tests.py --no-gui "nextcloudpi.local" 443 4443 || {
-            echo "Activation test failed!"
-            echo "Geckodriver logs:"
-            tail -n 20 geckodriver.log >&2 || true
-            echo "================"
-            echo "ncp.log: "
-            lxc exec ncp -- "tail -n20 /var/log/ncp.log"
-            exit 1
-          }
-          python system_tests.py --non-interactive || {
-            echo "System test failed!"
-            exit 1
-          }
-          python nextcloud_tests.py --no-gui "nextcloudpi.local" 443 4443 || {
+          export GECKODRIVER_PATH="$GECKODRIVER_PATH"
+          export FF_BINARY_PATH="$(which firefox)"
+          ../.venv/bin/python nextcloud_tests.py --no-gui --skip-release-check "nextcloudpi.local" 443 4443 || {
             echo "Nextcloud test failed!"
             echo "Geckodriver logs:"
             tail -n 20 geckodriver.log >&2 || true
             echo "================"
             echo "ncp.log: "
-            lxc exec ncp -- "tail -n20 /var/log/ncp.log"
+            sudo "$LXC" exec ncp -- "tail -n20 /var/log/ncp.log" || true
+            echo "================"
+            echo "nextcloud log: "
+            datadir="$(sudo "$LXC" exec ncp -- ncc config:system:get datadirectory)"
+            sudo "$LXC" exec ncp -- cat "$datadir/nextcloud.log" || true
             exit 1
           }
-          lxc stop ncp
+          sudo "$LXC" exec ncp -- bash -c 'tail -f /var/log/ncp.log' |& awk '{ print "NCP::" $0 }' &
+          USE_INCUS="$USE_INCUS" ../.venv/bin/python system_tests.py --non-interactive --skip-update-test || {
+            echo "System test failed!"
+            exit 1
+          }
+      - name: Checkout current version
+        run: |
+          git fetch origin
+          git checkout "${{ env.VERSION }}"
+      - name: NCP distupgrade
+        id: distupgrade
+        run: |
+          set -x
+          sudo "$LXC" exec ncp -- cat /etc/os-release | grep 'VERSION_ID="11"' || {
+            echo "can't upgrade from Debian $(sudo "$LXC" exec ncp -- cat /etc/os-release | grep VERSION_ID=)"
+            exit 1
+          }
+          current_nc_version="$(sudo "$LXC" exec ncp -- ncc status | grep "version:" | awk '{ print $3 }')"
+          latest_nc_version="$(cat etc/ncp.cfg | jq -r '.nextcloud_version')"
+          
+          sudo "$LXC" exec ncp -- bash -c "DEBIAN_FRONTEND=noninteractive ncp-dist-upgrade"
+          sudo "$LXC" exec ncp -- /usr/local/bin/ncc status
+          
+          if [[ "$current_nc_version" =~ "$latest_nc_version".* ]]
+          then
+            echo "Nextcloud is up to date - skipping NC update test."
+          else
+            
+            for i in {1..10};
+            do
+              echo "running nc update ($i/10)..."
+              sudo "$LXC" exec ncp -- bash -c "DBG=x ncp-update-nc ${latest_nc_version?}"
+            
+              sudo "$LXC" exec ncp -- /usr/local/bin/ncc status
+              current_nc_version_new="$(sudo "$LXC" exec ncp -- ncc status | grep "version:" | awk '{ print $3 }')"
+              if [[ "$current_nc_version_new" =~ "$latest_nc_version".* ]]
+              then
+                break
+              fi
+              if [[ "$current_nc_version" == "$current_nc_version_new" ]]
+              then
+                echo "failed to update to $latest_nc_version"
+                exit 1
+              fi
+              current_nc_version="$current_nc_version_new"
+            done
+          fi
+          
+          sudo "$LXC" exec ncp -- rm -f /opt/ncdata/data/nextcloud.log
+          sudo "$LXC" stop ncp
+      - name: Relaunch container
+        run: |
+          set -x
+          LXC_ARGS=()
+          [[ -z "$LXD_EXTRA_PROFILE" ]] || LXC_ARGS+=(-p "$LXD_EXTRA_PROFILE")
+          systemd-run --user --scope -p "Delegate=yes" "$LXC" start ncp || \
+          sudo systemd-run --scope -p "Delegate=yes" "$LXC" start ncp
+          sudo "$LXC" exec ncp -- bash -c 'while [ "$(systemctl is-system-running 2>/dev/null)" != "running" ] && [ "$(systemctl is-system-running 2>/dev/null)" != "degraded" ]; do :; done'
+          sleep 30
+          ip="$(sudo "$LXC" list -c n4 -f csv | grep '^ncp' | cut -d ',' -f2)"
+          ip="${ip/% *}"
+          sudo sed -i 's/^.*nextcloudpi.local.*$//g' /etc/hosts
+          echo "${ip} nextcloudpi.local" | sudo tee -a /etc/hosts
+      - name: Test LXD Image
+        working-directory: ./tests
+        run: |
+          set -x
+          export GECKODRIVER_PATH="$GECKODRIVER_PATH"
+          export FF_BINARY_PATH="$(which firefox)"
+          sudo "$LXC" exec ncp -- bash -c 'tail -f /var/log/ncp.log' |& awk '{ print "NCP::" $0 }' &
+          ../.venv/bin/python nextcloud_tests.py --no-gui "nextcloudpi.local" 443 4443 || {
+            echo "Nextcloud test failed!"
+            echo "Geckodriver logs:"
+            tail -n 20 geckodriver.log >&2 || true
+            echo "================"
+            echo "ncp.log: "
+            sudo "$LXC" exec ncp -- "tail -n20 /var/log/ncp.log" || true
+            echo "================"
+            echo "nextcloud log: "
+            datadir="$(sudo "$LXC" exec ncp -- ncc config:system:get datadirectory)"
+            sudo "$LXC" exec ncp -- cat "$datadir/nextcloud.log" || true
+            exit 1
+          }
+          USE_INCUS="$USE_INCUS" ../.venv/bin/python system_tests.py --non-interactive || {
+            echo "System test failed!"
+            exit 1
+          }
+          sudo "$LXC" stop ncp
 
   test-fresh-install:
     needs:
@@ -301,93 +625,122 @@ jobs:
       VERSION: "${{ inputs.git_ref || github.head_ref || github.ref_name }}"
       ARTIFACT_NAME: ${{ needs.build-current.outputs.artifact_name }}
       ARTIFACT_FILE: ${{ needs.build-current.outputs.artifact_file }}
+      LXC: "${{ needs.determine-runner.outputs.lxc_cmd }}"
+      USE_INCUS: "${{ needs.determine-runner.outputs.lxc_cmd == 'incus' && 'yes' || 'no' }}"
+      LXD_EXTRA_PROFILE: "${{ needs.determine-runner.outputs.lxd_extra_profile }}"
     steps:
       - name: Checkout code
         uses: actions/checkout@v3
         with:
           ref: "${{ env.VERSION }}"
-      - name: Cleanup lxd
-        run: test -z "$(lxc profile device show default | grep eth0)" || lxc profile device remove default eth0
-      - uses: whywaita/setup-lxd@v1
-        with:
-          lxd_version: latest/stable
-      - name: Fix LXD
-        continue-on-error: true
+      - name: Setup incus
+        if: ${{ needs.determine-runner.outputs.runner_label != 'ubuntu-20.04-arm64' }}
         run: |
-          lxc profile create network
-          cat <<EOF | lxc profile edit network
-          devices:
-            eth0:
-              name: eth0
-              network: lxdbr0
-              type: nic
-          EOF
+          sudo apt-get install -y incus qemu-system incus-tools
+          sudo iptables -I DOCKER-USER -i incusbr0 -j ACCEPT
+          sudo iptables -I DOCKER-USER -o incusbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
+          sudo incus admin init --auto
       - name: Setup Firefox
+        if: ${{ runner.arch != 'ARM64' }}
+        continue-on-error: true
+        id: setup-firefox-browser-action
         uses: browser-actions/setup-firefox@latest
-        continue-on-error: true
+      - name: Setup Firefox from packages
+        if: ${{ runner.arch == 'ARM64' || steps.setup-firefox-browser-action.outcome == 'failure' }}
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y --no-install-recommends firefox
       - name: Setup GeckoDriver
-        uses: ChlodAlejandro/setup-geckodriver@latest
-        continue-on-error: true
-        with:
-          token: ${{ secrets.GITHUB_TOKEN }}
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: |
+          set -x
+          arch=linux64
+          if [[ "${{ runner.arch }}" == "ARM64" ]]
+          then
+            arch="linux-aarch64"
+          fi
+          gh release download -R mozilla/geckodriver -p 'geckodriver-*-'"$arch.tar.gz"
+          tar xf "geckodriver-"*"-$arch.tar.gz"
+          sudo mv geckodriver /usr/local/bin/
+          sudo chmod +x /usr/local/bin/geckodriver
+          echo "GECKODRIVER_PATH=/usr/local/bin/geckodriver" >> "$GITHUB_ENV"
+          geckodriver -V
       - name: Setup Selenium
-        run: pip install selenium
+        run: |
+          sudo apt-get install -y python3-venv
+          python3 -m venv ./.venv
+          ./.venv/bin/pip install selenium
       - name: download LXD image from artifact store
-        uses: actions/download-artifact@v3
+        uses: actions/download-artifact@v4
         with:
           name: ${{ env.ARTIFACT_NAME }}
       - name: Remove old lxd images
         if: ${{ inputs.arch == 'arm64' }}
         run: |
-          for img in $(lxc image list -c f -f csv)
+          for img in $("$LXC" image list -c f -f csv)
           do
-            lxc image delete "$img"
+            sudo "$LXC" image delete "$img"
           done
       - name: Launch ncp container
         run: |
           set -x
           . ./build/buildlib.sh
-          lxc delete -q -f ncp || true
-          lxc image import -q "./${ARTIFACT_FILE?}" --alias "ncp/test" || true
-          systemd-run --user --scope -p "Delegate=yes" lxc launch -q "ncp/test" ncp || \
-          sudo systemd-run --scope -p "Delegate=yes" lxc launch -q "ncp/test" ncp 
-          lxc exec ncp -- bash -c 'while [ "$(systemctl is-system-running 2>/dev/null)" != "running" ] && [ "$(systemctl is-system-running 2>/dev/null)" != "degraded" ]; do :; done'
+          sudo "$LXC" delete -q -f ncp || true
+          sudo "$LXC" image import -q "./${ARTIFACT_FILE?}" --alias "ncp/test" || true
+          LXC_ARGS=(-p default)
+          [[ -z "$LXD_EXTRA_PROFILE" ]] || LXC_ARGS+=(-p "$LXD_EXTRA_PROFILE")
+          systemd-run --user --scope -p "Delegate=yes" "$LXC" launch -q "${LXC_ARGS[@]}" "ncp/test" ncp || \
+          sudo systemd-run --scope -p "Delegate=yes" "$LXC" launch -q "${LXC_ARGS[@]}" "ncp/test" ncp
+          sudo "$LXC" exec ncp -- bash -c 'while [ "$(systemctl is-system-running 2>/dev/null)" != "running" ] && [ "$(systemctl is-system-running 2>/dev/null)" != "degraded" ]; do :; done'
           sleep 30
-          ip="$(lxc list -c n4 -f csv | grep '^ncp' | cut -d ',' -f2)"
+          ip="$(sudo "$LXC" list -c n4 -f csv | grep '^ncp' | cut -d ',' -f2)"
           ip="${ip/% *}"
-          echo "${ip} nextcloudpi.local" | sudo tee /etc/hosts
+          sudo sed -i 's/^.*nextcloudpi.local.*$//g' /etc/hosts
+          echo "${ip} nextcloudpi.local" | sudo tee -a /etc/hosts
       - name: Test LXD Image
         working-directory: ./tests
         run: |
-          lxc exec ncp -- bash -c 'tail -f /var/log/ncp.log' |& awk '{ print "NCP::" $0 }' &
-          python activation_tests.py --no-gui "nextcloudpi.local" 443 4443 || {
+          set -x
+          export GECKODRIVER_PATH="$GECKODRIVER_PATH"
+          export FF_BINARY_PATH="$(which firefox)"
+          sudo "$LXC" exec ncp -- bash -c 'tail -f /var/log/ncp.log' |& awk '{ print "NCP::" $0 }' &
+          ../.venv/bin/python activation_tests.py --no-gui "nextcloudpi.local" 443 4443 || {
             echo "Activation test failed!"
             echo "Geckodriver logs:"
             tail -n 20 geckodriver.log >&2 || true
             echo "================"
             echo "ncp.log: "
-            lxc exec ncp -- "tail -n20 /var/log/ncp.log"
+            sudo "$LXC" exec ncp -- "tail -n20 /var/log/ncp.log" || true
+            echo "================"
+            echo "nextcloud log: "
+            datadir="$("$LXC" exec ncp -- ncc config:system:get datadirectory)"
+            sudo "$LXC" exec ncp -- cat "$datadir/nextcloud.log" || true
             exit 1
           }
-          python system_tests.py --non-interactive || {
-            echo "System test failed!"
-            exit 1
-          }
-          python nextcloud_tests.py --no-gui "nextcloudpi.local" 443 4443 || {
+          ../.venv/bin/python nextcloud_tests.py --no-gui "nextcloudpi.local" 443 4443 || {
             echo "Nextcloud test failed!"
             echo "Geckodriver logs:"
             tail -n 20 geckodriver.log >&2 || true
             echo "================"
             echo "ncp.log: "
-            lxc exec ncp -- "tail -n20 /var/log/ncp.log"
+            sudo "$LXC" exec ncp -- "tail -n20 /var/log/ncp.log" || true
+            echo "================"
+            echo "nextcloud log: "
+            datadir="$(sudo "$LXC" exec ncp -- ncc config:system:get datadirectory)"
+            sudo "$LXC" exec ncp -- cat "$datadir/nextcloud.log" || true
             exit 1
           }
-          lxc stop ncp
+          USE_INCUS="$USE_INCUS" ../.venv/bin/python system_tests.py --non-interactive || {
+            echo "System test failed!"
+            exit 1
+          }
+          sudo "$LXC" stop ncp
 
   convert-to-lxc-image:
     needs:
       - build-current
-    runs-on: ubuntu-20.04
+    runs-on: ubuntu-latest
     outputs:
       artifact_name: "${{ steps.lxd-to-lxc.outputs.artifact_name }}"
       artifact_file: "${{ steps.lxd-to-lxc.outputs.artifact_file }}"
@@ -397,24 +750,25 @@ jobs:
       LXD_ARTIFACT_FILE: ${{ needs.build-current.outputs.artifact_file }}
     steps:
       - name: download LXD image from artifact store
-        uses: actions/download-artifact@v3
+        uses: actions/download-artifact@v4
         with:
           name: ${{ env.LXD_ARTIFACT_NAME }}
       - name: convert to LXC image
         id: lxd-to-lxc
         run: |
-          mkdir repackage output
-          cd repackage
-          sudo tar xpf "../${LXD_ARTIFACT_FILE?}"
-          sudo rm -rf ./rootfs/dev
-          LXC_ARTIFACT_FILE="${LXD_ARTIFACT_FILE//LXD/LXC}"
-          sudo tar cpzf "../output/${LXC_ARTIFACT_FILE?}" -C rootfs/ .
-          cd ..
-          sudo chown "$(id -un):" "./output/${LXC_ARTIFACT_FILE}"
-          echo "artifact_file=${LXC_ARTIFACT_FILE?}" >> $GITHUB_OUTPUT
-          echo "artifact_name=${LXD_ARTIFACT_NAME//lxd/lxc}" >> $GITHUB_OUTPUT
+          mkdir repackage output;
+          cd repackage; 
+          sudo su - -c "cd '$(pwd)'; tar xpf '../${LXD_ARTIFACT_FILE?}';"
+          sudo rm -rf ./rootfs/dev;
+          LXC_ARTIFACT_FILE="${LXD_ARTIFACT_FILE//LXD/LXC}";
+          sudo tar cpzf "../output/${LXC_ARTIFACT_FILE?}" -C rootfs/ .;
+          cd ..;
+          sudo chown "$(id -un):" "./output/${LXC_ARTIFACT_FILE}";  
+          ls -l ./output
+          echo "artifact_file=${LXC_ARTIFACT_FILE?}" >> $GITHUB_OUTPUT;
+          echo "artifact_name=${LXD_ARTIFACT_NAME//lxd/lxc}" >> $GITHUB_OUTPUT;
       - name: upload LXD image to artifact store
-        uses: actions/upload-artifact@v3
+        uses: actions/upload-artifact@v4
         with:
           name: "${{ steps.lxd-to-lxc.outputs.artifact_name }}"
           path: "output/${{ steps.lxd-to-lxc.outputs.artifact_file }}"

.github/workflows/build-sd-images.yml

@@ -15,6 +15,10 @@ on:
         required: false
         default: true
         type: boolean
+      test_image_url:
+        required: false
+        type: string
+        default: ""
     outputs:
       artifact_name:
         value: "${{ jobs.build.outputs.artifact_name }}"
@@ -23,7 +27,7 @@ on:
 
 jobs:
   build:
-    runs-on: ubuntu-20.04
+    runs-on: ubuntu-latest
     env:
       VERSION: "${{ inputs.git_ref }}"
       LOG_GUEST: "\\033[1;34mGUEST::\\033[0m"
@@ -39,10 +43,15 @@ jobs:
       artifact_name: ${{ github.run_id }}-${{ inputs.board_id }}-image
     steps:
       - name: Set up QEMU
-        uses: docker/setup-qemu-action@v2
-      - name: Setup qemu-user-static
-        run: |
-          sudo apt-get update
+        uses: docker/setup-qemu-action@v3
+#      - name: Set up QEMU
+#        run: |
+#          sudo apt-get update
+#          sudo apt-get install -y binfmt-support
+#          curl -L -o /tmp/qemu.sh 'https://raw.githubusercontent.com/qemu/qemu/master/scripts/qemu-binfmt-conf.sh'
+#          bash /tmp/qemu.sh --debian
+#          sudo systemctl disable apparmor
+#          docker run --rm --privileged tonistiigi/binfmt:latest --install all
 #          docker run --rm --privileged multiarch/qemu-user-static:register --reset --credential yes
 #          sudo mkdir -p /etc/binfmt
 #          for conf in qemu-{aarch64,arm}-static.conf
@@ -58,33 +67,36 @@ jobs:
 #          which qemu-aarch64-static
 #          update-binfmts --display qemu-aarch64
 #          update-binfmts --display qemu-arm
+      - name: "Skip build"
+        if: "${{ inputs.test_image_url != '' }}"
+        run: |
+          echo "artifact_file=NextcloudPi_TestImage_${VERSION//\//_}.img" >> $GITHUB_OUTPUT
+          echo "ARTIFACT_FILE=NextcloudPi_TestImage_${VERSION//\//_}.img" >> $GITHUB_ENV
       - name: "Build Armbian"
-        if: ${{ inputs.board_id != 'raspberrypi' }}
+        if: "${{ inputs.test_image_url == '' }}"
         id: build-armbian
         continue-on-error: true
         run: |
           set -ex
-          export LIB_TAG=master
-          export IMG="NextCloudPi_${{ inputs.board_name }}_${VERSION//\//_}.img"
+          export IMG="NextcloudPi_${{ inputs.board_name }}_${VERSION//\//_}.img"
           [[ "${{ github.ref_protected }}" == true ]] || export DBG=x
-          
+
           ./build/build-SD-armbian.sh "${{ inputs.board_id }}" "${{ inputs.board_name }}"
-          
+
           artifacts=("armbian/output/images/Armbian"*.img)
           mkdir -p output
           mv "${artifacts[0]}" "output/$IMG"
           echo "artifact_file=${IMG}" >> $GITHUB_OUTPUT
           echo "ARTIFACT_FILE=${IMG}" >> $GITHUB_ENV
       - name: "Build Armbian (2nd attempt)"
-        if: ${{ inputs.board_id != 'raspberrypi' && steps.build-armbian.outcome == 'failure' }}
+        if: ${{ steps.build-armbian.outcome == 'failure' }}
         id: build-armbian-2nd
         run: |
           set -ex
           echo -e "${LOG_CICD} Cleanup armbian build leftovers..."
           sudo rm -rf armbian/ tmp/ output/
-          
-          export LIB_TAG=master
-          export IMG="NextCloudPi_${{ inputs.board_name }}_${VERSION//\//_}.img"
+
+          export IMG="NextcloudPi_${{ inputs.board_name }}_${VERSION//\//_}.img"
           [[ "${{ github.ref_protected }}" == true ]] || export DBG=x
 
           ./build/build-SD-armbian.sh "${{ inputs.board_id }}" "${{ inputs.board_name }}"
@@ -95,36 +107,14 @@ jobs:
           echo "artifact_file=${IMG}" >> $GITHUB_OUTPUT
           echo "ARTIFACT_FILE=${IMG}" >> $GITHUB_ENV
       - name: "Upload Armbian logs"
-        if: ${{ inputs.board_id != 'raspberrypi' && failure() }}
-        uses: actions/upload-artifact@v3
+        if: ${{ failure() }}
+        uses: actions/upload-artifact@v4
         with:
           name: ${{ github.run_id }}-${{ inputs.board_id }}-logs
           path: armbian/output
-      - name: Build RPI SD Image
-        if: ${{ inputs.board_id == 'raspberrypi' }}
-        id: build-rpi
-        run: |
-          set -ex
-          echo -e "${LOG_CICD} Protected? ${{ github.ref_protected }}"
-          export IMG="NextCloudPi_${{ inputs.board_name }}_${VERSION//\//_}.img"
-          [[ "${{ github.ref_protected }}" == true ]] || export DBG=x
-          wget -q https://github.com/multiarch/qemu-user-static/releases/latest/download/qemu-aarch64-static -O ./qemu-aarch64-static
-          ./build/build-SD-rpi.sh
-          mkdir -p output
-          mv "tmp/$IMG" ./output/
-
-          for i in {1..10}
-          do
-            sudo losetup | grep "${IMG}" || break;
-            [[ "$i" -lt 10 ]] || { echo -e "${LOG_CICD} Timeout while waiting for image to unmount"; exit 1; }
-            sleep 6
-            echo -e "${LOG_CICD} Retrying ($i out of 10)"
-          done
-
-          echo "artifact_file=${IMG}" >> $GITHUB_OUTPUT
-          echo "ARTIFACT_FILE=${IMG}" >> $GITHUB_ENV
       - name: upload image to artifact store
-        uses: actions/upload-artifact@v3
+        if: "${{ inputs.test_image_url == '' }}"
+        uses: actions/upload-artifact@v4
         with:
           name: ${{ github.run_id }}-${{ inputs.board_id }}-image
           path: output/${{ env.ARTIFACT_FILE }}
@@ -132,7 +122,7 @@ jobs:
 
   test:
     needs: build
-    runs-on: ubuntu-20.04
+    runs-on: ubuntu-22.04
     env:
       VERSION: "${{ inputs.git_ref }}"
       ARTIFACT_ID: ${{ needs.build.outputs.artifact_name }}
@@ -147,42 +137,51 @@ jobs:
         shell: bash
     steps:
       - name: Set up QEMU
-        uses: docker/setup-qemu-action@v2
-      - name: Apt update
+        uses: docker/setup-qemu-action@v3
+      - name: Disable apparmor
         run: |
-          sudo apt-get update
-#          sudo apt-get -y --no-install-recommends install qemu-user-static
-#      - name: Apply workaround for sudo bug (https://github.com/multiarch/qemu-user-static/issues/17)
-#        run: |
-#          sudo apt-get update
-#          sudo apt-get -y --no-install-recommends install binfmt-support qemu-user-static
-#          docker run --rm --privileged multiarch/qemu-user-static:register --reset --credential yes
-#          sudo mkdir -p /etc/binfmt
-#          for conf in qemu-{aarch64,arm}-static.conf
-#          do
-#            sed 's/:F$/:OC/' /usr/lib/binfmt.d/$conf | sudo tee /etc/binfmt/$conf
-#          done
+          sudo systemctl disable apparmor
       - name: Checkout code
         uses: actions/checkout@v3
         with:
           ref: "${{ env.VERSION }}"
-      - uses: actions/download-artifact@v3
+      - uses: actions/download-artifact@v4
+        if: "${{ inputs.test_image_url == '' }}"
         with:
           name: ${{ env.ARTIFACT_ID }}
           path: output
+      - name: Download Test Image
+        if: "${{ inputs.test_image_url != '' }}"
+        run: |
+          set -ex
+          mkdir -p output
+          sudo apt-get install -y pv
+          wget -nv "${{ inputs.test_image_url }}" -O - | pv -n --bytes --rate --timer -i 3 > "output/${ARTIFACT_FILE?}"
       - name: Prepare test
         run: |
           set -x
-          mv output/${ARTIFACT_FILE?} ncp.img
+          mv "output/${ARTIFACT_FILE?}" ncp.img
           sudo apt-get install -y systemd-container
-          sudo pip install selenium
+          python3 -m venv ./.venv
+          . ./.venv/bin/activate
+          ./.venv/bin/pip install selenium
           sudo rm -rf raspbian_root
           . ./build/buildlib.sh
           mount_raspbian "ncp.img"
+          sudo cat raspbian_root/etc/machine-id
+          sudo systemd-id128 new | sudo tee ./raspbian_root/etc/machine-id
           sudo wget -nv https://github.com/multiarch/qemu-user-static/releases/latest/download/qemu-aarch64-static -O raspbian_root/usr/bin/qemu-aarch64-static
           sudo wget -nv https://github.com/multiarch/qemu-user-static/releases/latest/download/qemu-arm-static -O raspbian_root/usr/bin/qemu-arm-static
           sudo chmod +x raspbian_root/usr/bin/qemu-{arm,aarch64}-static
           echo 'Mutex posixsem' | sudo tee -a raspbian_root/etc/apache2/mods-available/ssl.conf
+          echo 'ignore-warnings ARM64-COW-BUG' | sudo tee -a raspbian_root/etc/redis/redis.conf
+          sudo mkdir -p raspbian_root/etc/systemd/system/redis-server.service.d
+          echo '[Service]' | sudo tee raspbian_root/etc/systemd/system/redis-server.service.d/ncp.conf
+          echo 'PrivateUsers=false' | sudo tee -a raspbian_root/etc/systemd/system/redis-server.service.d/ncp.conf
+          
+          sudo mkdir -p raspbian_root/etc/systemd/system/php8.3-fpm.service.d
+          echo '[Service]' | sudo tee raspbian_root/etc/systemd/system/php8.3-fpm.service.d/ncp.conf
+          echo 'ExecStartPre=mkdir -p /var/run/php' | sudo tee -a raspbian_root/etc/systemd/system/php8.3-fpm.service.d/ncp.conf
       - name: Test image
         id: test
         run: |
@@ -203,7 +202,7 @@ jobs:
           for attempt in {1..30}
           do
             echo -e "${LOG_CICD} == Wait until container network is available (attempt $attempt/30) =="
-            ip="$("${CONTAINER_CMD[@]}" bash -c '. /usr/local/etc/library.sh > /dev/null; get_ip')"
+                ip="$("${CONTAINER_CMD[@]}" bash -c '. /usr/local/etc/library.sh > /dev/null; get_ip')"
             [[ -n "$ip" ]] && curl -k "https://$ip/activate/" > /dev/null || { sleep 6; continue; }
             success=true
             break
@@ -238,12 +237,14 @@ jobs:
           done
           
           [[ "$success" == "true" ]] || {
-              echo -e "${LOG_CICD} Timeout reached."
-              "${CONTAINER_CMD[@]}" -q systemctl status mysql |& awk "{ print \"${LOG_DIAG} \" \$0 }"
-              "${CONTAINER_CMD[@]}" -q systemctl status redis |& awk "{ print \"${LOG_DIAG} \" \$0 }"
-              "${CONTAINER_CMD[@]}" -q systemctl status 'php*-fpm' |& awk "{ print \"${LOG_DIAG} \" \$0 }"
-              "${CONTAINER_CMD[@]}" -q systemctl status apache2 |& awk "{ print \"${LOG_DIAG} \" \$0 }"
-              "${CONTAINER_CMD[@]}" -q ncp-diag |& awk "{ print \"${LOG_DIAG} \" \$0 }"
+              echo -e "${LOG_CICD} ERR: Timeout reached."
+              "${CONTAINER_CMD[@]}" -q systemctl status mysql |& awk "{ print \"${LOG_DIAG} \" \$0 }" ||:
+              "${CONTAINER_CMD[@]}" -q systemctl status redis-server |& awk "{ print \"${LOG_DIAG} \" \$0 }" ||:
+              sudo journalctl --file ./raspbian_root/var/log/journal/"$(sudo cat ./raspbian_root/etc/machine-id)"/system.journal --no-pager -eu redis-server ||:
+              "${CONTAINER_CMD[@]}" -q journalctl --no-pager -eu redis-server
+              "${CONTAINER_CMD[@]}" -q systemctl status 'php*-fpm' |& awk "{ print \"${LOG_DIAG} \" \$0 }" ||:
+              "${CONTAINER_CMD[@]}" -q systemctl status apache2 |& awk "{ print \"${LOG_DIAG} \" \$0 }" ||:
+              "${CONTAINER_CMD[@]}" -q ncp-diag |& awk "{ print \"${LOG_DIAG} \" \$0 }" ||:
               exit 1
           }
           
@@ -254,7 +255,7 @@ jobs:
           for attempt in {1..5}
           do
             echo -e "${LOG_CICD} == Activation Tests (attempt $attempt/5) =="
-            python tests/activation_tests.py -t 300 --no-gui "$ip" 443 4443 |& awk "{ print \"${LOG_TEST} \" \$0 }"
+            ./.venv/bin/python tests/activation_tests.py -t 300 --no-gui "$ip" 443 4443 |& awk "{ print \"${LOG_TEST} \" \$0 }"
             [[ ${PIPESTATUS[0]} -eq 0 ]] || {
               echo -e "${LOG_CICD} Activation test failed!"
               echo -e "${LOG_DIAG} Geckodriver logs:"
@@ -278,7 +279,7 @@ jobs:
           for attempt in {1..5}
           do
             echo -e "${LOG_CICD} == System Tests (attempt $attempt/5) =="
-            sudo python tests/system_tests.py --non-interactive |& awk "{ print \"${LOG_TEST} \" \$0 }"
+            USE_SUDO=yes ./.venv/bin/python tests/system_tests.py --non-interactive |& awk "{ print \"${LOG_TEST} \" \$0 }"
             [[ ${PIPESTATUS[0]} -eq 0 ]] || {
               echo -e "${LOG_CICD} System test failed!"
               sleep 12
@@ -296,14 +297,27 @@ jobs:
           for attempt in {1..5}
           do
             echo -e "${LOG_CICD} == Nextcloud Tests (attempt $attempt/5) =="
-            python tests/nextcloud_tests.py --no-gui "$ip" 443 4443 |& awk "{ print \"${LOG_TEST} \" \$0 }"
+            ./.venv/bin/python tests/nextcloud_tests.py --no-gui "$ip" 443 4443 |& awk "{ print \"${LOG_TEST} \" \$0 }"
             [[ ${PIPESTATUS[0]} -eq 0 ]] || {
               echo -e "${LOG_CICD} Nextcloud test failed!"
+              echo -e "${LOG_DIAG} /etc/os-release:"
+              "${CONTAINER_CMD[@]}" -q ncp /bin/bash -c 'cat /etc/os-release'
+              echo -e "${LOG_DIAG} /usr/local/etc/ncp.cfg:"
+              "${CONTAINER_CMD[@]}" --pipe -q ncp /bin/bash -c 'cat /usr/local/etc/ncp.cfg'
+              cat ./raspbian_root/usr/local/etc/ncp.cfg
+              echo -e "${LOG_DIAG} /home/ncp-app-bridge config ncp"
+              sudo ls -l ./raspbian_root/home/www/ncp-app-bridge.sh
+              "${CONTAINER_CMD[@]}" --pipe --uid=33 ncp /bin/bash -c 'sudo /home/www/ncp-app-bridge.sh config ncp'
               echo -e "{$LOG_DIAG} Geckodriver logs:"
               tail -n 20 geckodriver.log >&2 |& awk "{ print \"${LOG_DIAG} \" \$0 }" || true
               echo -e "${LOG_CICD} ================"
               echo -e "${LOG_DIAG} ncp.log: "
-              "${CONTAINER_CMD[@]}" -q ncp /bin/bash -c "tail -n20 /var/log/ncp.log" |& awk "{ print \"${LOG_DIAG} \" \$0 }" || true
+              "${CONTAINER_CMD[@]}" --pipe ncp /bin/bash -c "tail -n20 /var/log/ncp.log" |& awk "{ print \"${LOG_DIAG} \" \$0 }" || true
+              echo "================"
+              echo "${LOG_DIAG} Nextcloud log: "
+              "${CONTAINER_CMD[@]}" --pipe -q ncp /bin/bash -c 'ls -l /opt/ncdata/data/nextcloud.log' |& awk "{ print \"${LOG_DIAG} \" \$0 }" || true
+              "${CONTAINER_CMD[@]}" --pipe -q ncp /bin/bash -c 'cat /opt/ncdata/data/nextcloud.log' |& awk "{ print \"${LOG_DIAG} \" \$0 }" || true
+              sudo cat ./raspbian_root/opt/ncdata/data/nextcloud.log |& awk "{ print \"${LOG_DIAG} \" \$0 }"
               sleep 12
               continue
             }
@@ -315,5 +329,541 @@ jobs:
           
           [[ "$success" == "true" ]] || {
             echo -e "${LOG_CICD} Nextcloud test failed in all attempts!"
+          
+            echo "Apache2 logs (error):"
+            sudo cat ./raspbian_root/var/log/apache2/error.log || true
+            echo "===================="
+            
+            echo "Apache2 logs (nc-error):"
+            sudo cat ./raspbian_root/var/log/apache2/nc-error.log || true
+            echo "===================="
+            
+            echo "PHP FPM logs:"
+            sudo cat ./raspbian_root/var/log/php*-fpm.log
+            echo "===================="
+            
+            echo "Nextcloud logs:"
+            sudo cat ./raspbian_root/opt/ncdata/data/nextcloud.log
+            echo "===================="
+            
+            echo "MySQL logs:"
+            sudo journalctl --file ./raspbian_root/var/log/journal/"$(sudo cat ./raspbian_root/etc/machine-id)"/system.journal --no-pager -eu mariadb || true
+            echo "===================="
+            
+            echo "Redis logs:"
+            sudo journalctl --file ./raspbian_root/var/log/journal/"$(sudo cat ./raspbian_root/etc/machine-id)"/system.journal --no-pager -eu redis-server || true
+            echo "===================="
             exit 1
           }
+
+#  test-distupgrade:
+#    runs-on: ubuntu-latest
+#    env:
+#      VERSION: "${{ inputs.git_ref || github.ref }}"
+#      LOG_GUEST: "\\033[1;34mGUEST::\\033[0m"
+#      LOG_NCP: "\\033[1;36m~NCP::\\033[0m"
+#      LOG_CICD: "\\033[1;35mCICD::\\033[0m"
+#      LOG_TEST: "\\033[1;33mTEST::\\033[0m"
+#      LOG_DIAG: "\\033[1;31mDIAG::\\033[0m"
+#      PREVIOUS_VERSION: "v1.53.2"
+#    defaults:
+#      run:
+#        shell: bash
+#    steps:
+#      - name: Set up QEMU
+#        uses: docker/setup-qemu-action@v3
+#      - name: Checkout code
+#        uses: actions/checkout@v3
+#        with:
+#          ref: "${{ env.VERSION }}"
+#      - name: Download previous image
+#        id: download-previous-image
+#        run: |
+#          set -x
+#          mkdir -p output
+#          cd output
+#          wgetrc=0
+#          wgeterr="$(wget -O ./ncp.zip "https://github.com/nextcloud/nextcloudpi/releases/download/${PREVIOUS_VERSION?}/NextcloudPi_${{ inputs.board_name }}_${PREVIOUS_VERSION}.zip" 2>&1)" || wgetrc=$?
+#          if [[ $wgetrc -ne 0 ]]
+#          then
+#            if echo "$wgeterr" | grep '404 Not Found'
+#            then
+#              echo "Board not found in previous release - skipping."
+#              echo "skipped=true" >> "$GITHUB_OUTPUT"
+#              exit 0
+#            else
+#              echo "$wgeterr"
+#              exit $wgetrc
+#            fi
+#          fi
+#          echo "skipped=false" >> "$GITHUB_OUTPUT"
+#          unzip ncp.zip
+#          rm ncp.zip
+#          mv NextcloudPi_${{ inputs.board_name }}_${PREVIOUS_VERSION}.img ./ncp.img
+#          echo "ARTIFACT_FILE=ncp.img" >> "$GITHUB_ENV"
+#      - name: Prepare test
+#        if:  ${{ steps.download-previous-image.outputs.skipped == 'false' }}
+#        run: |
+#          set -x
+#          mv output/${ARTIFACT_FILE?} ncp.img
+#          sudo apt-get install -y systemd-container
+#          sudo pip install selenium
+#          sudo rm -rf raspbian_root
+#          . ./build/buildlib.sh
+#          mount_raspbian "ncp.img"
+#          sudo cat raspbian_root/etc/machine-id
+#          sudo systemd-id128 new | sudo tee ./raspbian_root/etc/machine-id
+#          sudo wget -nv https://github.com/multiarch/qemu-user-static/releases/latest/download/qemu-aarch64-static -O raspbian_root/usr/bin/qemu-aarch64-static
+#          sudo wget -nv https://github.com/multiarch/qemu-user-static/releases/latest/download/qemu-arm-static -O raspbian_root/usr/bin/qemu-arm-static
+#          sudo chmod +x raspbian_root/usr/bin/qemu-{arm,aarch64}-static
+#          echo 'Mutex posixsem' | sudo tee -a raspbian_root/etc/apache2/mods-available/ssl.conf
+#          echo 'ignore-warnings ARM64-COW-BUG' | sudo tee -a raspbian_root/etc/redis/redis.conf
+#          sudo mkdir -p raspbian_root/etc/systemd/system/redis-server.service.d
+#          echo '[Service]' | sudo tee raspbian_root/etc/systemd/system/redis-server.service.d/ncp.conf
+#          echo 'PrivateUsers=false' | sudo tee -a raspbian_root/etc/systemd/system/redis-server.service.d/ncp.conf
+#      - name: Test and activate image
+#        if:  ${{ steps.download-previous-image.outputs.skipped == 'false' }}
+#        id: test
+#        run: |
+#
+#          log_err() {
+#            rc="${1?}"
+#            msg="${2?}"
+#            echo -e "${LOG_DIAG} $msg" >&2
+#            return $rc
+#          }
+#
+#          sudo systemd-nspawn --boot -D ./raspbian_root/ -M ncp --hostname=nextcloudpi |& awk "{ print \"${LOG_GUEST} \" \$0 }" &
+#          sleep 60
+#
+#          CONTAINER_CMD=(sudo systemd-run --machine=ncp -P --wait)
+#
+#          success=false
+#          for attempt in {1..30}
+#          do
+#            echo -e "${LOG_CICD} == Wait until container network is available (attempt $attempt/30) =="
+#            ip="$("${CONTAINER_CMD[@]}" bash -c '. /usr/local/etc/library.sh > /dev/null; get_ip')"
+#            [[ -n "$ip" ]] && curl -k "https://$ip/activate/" > /dev/null || { sleep 6; continue; }
+#            success=true
+#            break
+#          done
+#          sudo cat ./raspbian_root/var/log/ncp.log |& awk "{ print \"${LOG_NCP} \" \$0 }"
+#          sudo tail -n 0 -f ./raspbian_root/var/log/ncp.log |& awk "{ print \"${LOG_NCP} \" \$0 }" &
+#
+#          [[ "$success" == "true" ]] || {
+#            echo -e "${LOG_CICD} Could not reach container. Aborting..."
+#            exit 1
+#          }
+#
+#          attempt=0
+#          success=false
+#          for attempt in {1..150}
+#          do
+#            echo -e "${LOG_CICD} Waiting for container startup (attempt $attempt/150)..."
+#            "${CONTAINER_CMD[@]}" journalctl -eu redis.service || true
+#            "${CONTAINER_CMD[@]}" systemctl status php8.1-fpm.service || true
+#            redis_pw="$("${CONTAINER_CMD[@]}" bash -c ". /usr/local/etc/library.sh; get_nc_config_value 'redis\"][\"password'")" \
+#            && redis_socket="$("${CONTAINER_CMD[@]}" bash -c ". /usr/local/etc/library.sh; get_nc_config_value 'redis\"][\"host'")" \
+#            || log_err $? "Error retrieving redis credentials" || true
+#            if { "${CONTAINER_CMD[@]}" -q ncc status |& awk "{ print \"${LOG_DIAG} \" \$0 }" || log_err $? "ncc status check failed"; } \
+#              && { [[ "$("${CONTAINER_CMD[@]}" ncc maintenance:mode)" =~ .*disabled.* ]] || log_err $? "Maintenance mode is enabled or could not be retrieved"; } \
+#              && { "${CONTAINER_CMD[@]}" redis-cli -s "$redis_socket" -a "$redis_pw" set redisready yes |& awk "{ print \"${LOG_DIAG} \" \$0 }" || log_err $? "Failed to set redis variable"; } \
+#              && { "${CONTAINER_CMD[@]}" redis-cli -s "$redis_socket" -a "$redis_pw" get redisready |& awk "{ print \"${LOG_DIAG} \" \$0 }" || log_err $? "Failed to read redis variable"; }
+#            then
+#              echo -e "${LOG_CICD} Startup successful"
+#              success=true
+#              break
+#            fi
+#            attempt=$((attempt + 1))
+#            sleep 5
+#          done
+#
+#          [[ "$success" == "true" ]] || {
+#              echo -e "${LOG_CICD} Timeout reached."
+#              "${CONTAINER_CMD[@]}" -q systemctl status mysql |& awk "{ print \"${LOG_DIAG} \" \$0 }"
+#              "${CONTAINER_CMD[@]}" -q systemctl status redis |& awk "{ print \"${LOG_DIAG} \" \$0 }"
+#              "${CONTAINER_CMD[@]}" -q systemctl status 'php*-fpm' |& awk "{ print \"${LOG_DIAG} \" \$0 }"
+#              "${CONTAINER_CMD[@]}" -q systemctl status apache2 |& awk "{ print \"${LOG_DIAG} \" \$0 }"
+#              "${CONTAINER_CMD[@]}" -q ncp-diag |& awk "{ print \"${LOG_DIAG} \" \$0 }"
+#              exit 1
+#          }
+#
+#          set -x
+#          set +e
+#
+#          success=false
+#          for attempt in {1..5}
+#          do
+#            echo -e "${LOG_CICD} == Activation Tests (attempt $attempt/5) =="
+#            python tests/activation_tests.py -t 300 --no-gui "$ip" 443 4443 |& awk "{ print \"${LOG_TEST} \" \$0 }"
+#            [[ ${PIPESTATUS[0]} -eq 0 ]] || {
+#              echo -e "${LOG_CICD} Activation test failed!"
+#              echo -e "${LOG_DIAG} Geckodriver logs:"
+#              tail -n 20 geckodriver.log >&2 |& awk "{ print \"${LOG_DIAG} \" \$0 }" || true
+#              echo "================"
+#              echo -e "${LOG_DIAG} mysql: "
+#              "${CONTAINER_CMD[@]}" -q ncp-diag |& awk "{ print \"${LOG_DIAG} \" \$0 }" || true
+#              "${CONTAINER_CMD[@]}" -q systemctl status mysql |& awk "{ print \"${LOG_DIAG} \" \$0 }" || true
+#              sleep 12
+#              continue
+#            }
+#            success=true
+#            break
+#          done
+#          [[ "$success" == "true" ]] || {
+#            echo -e "${LOG_CICD} Activation test failed in all attempts!"
+#            exit 1
+#          }
+#
+#          success=false
+#          for attempt in {1..5}
+#          do
+#            echo -e "${LOG_CICD} == System Tests (attempt $attempt/5) =="
+#            sudo python tests/system_tests.py --non-interactive --skip-update-test |& awk "{ print \"${LOG_TEST} \" \$0 }"
+#            [[ ${PIPESTATUS[0]} -eq 0 ]] || {
+#              echo -e "${LOG_CICD} System test failed!"
+#              sleep 12
+#              continue
+#            }
+#            success=true
+#            break
+#          done
+#          [[ "$success" == "true" ]] || {
+#            echo -e "${LOG_CICD} System test failed in all attempts!"
+#            exit 1
+#          }
+#
+#          success=false
+#          for attempt in {1..5}
+#          do
+#            echo -e "${LOG_CICD} == Nextcloud Tests (attempt $attempt/5) =="
+#            python tests/nextcloud_tests.py --no-gui --skip-release-check "$ip" 443 4443 |& awk "{ print \"${LOG_TEST} \" \$0 }"
+#            [[ ${PIPESTATUS[0]} -eq 0 ]] || {
+#              echo -e "${LOG_CICD} Nextcloud test failed!"
+#              echo -e "{$LOG_DIAG} Geckodriver logs:"
+#              tail -n 20 geckodriver.log >&2 |& awk "{ print \"${LOG_DIAG} \" \$0 }" || true
+#              echo -e "${LOG_CICD} ================"
+#              echo -e "${LOG_DIAG} ncp.log: "
+#              "${CONTAINER_CMD[@]}" -q ncp /bin/bash -c "tail -n20 /var/log/ncp.log" |& awk "{ print \"${LOG_DIAG} \" \$0 }" || true
+#              echo "================"
+#              echo "${LOG_DIAG} Nextcloud log: "
+#              "${CONTAINER_CMD[@]}" -q ncp /bin/bash -c 'ls -l /opt/ncdata/data/nextcloud.log' |& awk "{ print \"${LOG_DIAG} \" \$0 }" || true
+#              "${CONTAINER_CMD[@]}" -q ncp /bin/bash -c 'cat /opt/ncdata/data/nextcloud.log' |& awk "{ print \"${LOG_DIAG} \" \$0 }" || true
+#              sleep 12
+#              continue
+#            }
+#            success=true
+#            break
+#          done
+#
+#          [[ "$success" == "true" ]] || {
+#            echo -e "${LOG_CICD} Nextcloud test failed in all attempts!"
+#            exit 1
+#          }
+#      - name: Update NCP
+#        if:  ${{ steps.download-previous-image.outputs.skipped == 'false' }}
+#        run: |
+#          set -ex
+#
+#          CONTAINER_CMD=(sudo systemd-run --machine=ncp -P --wait)
+#
+#          BRANCH="${VERSION/refs\/heads\//}"
+#          BRANCH="${BRANCH/refs\/tags\//}"
+#          if [[ "$BRANCH" =~ "refs/pull/"* ]]
+#          then
+#            UPDATE_ARGS=("${{ github.head_ref }}" "$VERSION")
+#          else
+#            UPDATE_ARGS=("$BRANCH")
+#          fi
+#          current_nc_version="$("${CONTAINER_CMD[@]}" ncc status | grep "version:" | awk '{ print $3 }')"
+#          latest_nc_version="$(cat etc/ncp.cfg | jq -r '.nextcloud_version')"
+#
+#          echo "Updating from $PREVIOUS_VERSION to $VERSION"
+#
+#          "${CONTAINER_CMD[@]}" bash -c "DBG=x ncp-update ${UPDATE_ARGS[*]}"
+#          "${CONTAINER_CMD[@]}" /usr/local/bin/ncc status
+#          # "${CONTAINER_CMD[@]}" bash -c 'curl https://download.nextcloud.com/server/releases/nextcloud-28.0.4.tar.bz2 > /var/www/nextcloud-28.0.4.tar.bz2'
+#          k0nKat1Nation
+#
+#          # if [[ "$current_nc_version" =~ "$latest_nc_version".* ]]
+#          # then
+#          #   echo "Nextcloud is up to date - skipping NC update test."
+#          # else
+#          #   "${CONTAINER_CMD[@]}" bash -c "DBG=x ncp-update-nc ${latest_nc_version?}"
+#          # fi
+#
+#          sudo machinectl terminate ncp
+#          sudo rm -f ./raspbian_root/opt/ncdata/data/nextcloud.log
+#      - name: Test image after update
+#        if:  ${{ steps.download-previous-image.outputs.skipped == 'false' }}
+#        run: |
+#
+#          log_err() {
+#            rc="${1?}"
+#            msg="${2?}"
+#            echo -e "${LOG_DIAG} $msg" >&2
+#            return $rc
+#          }
+#
+#          sudo systemd-nspawn --boot -D ./raspbian_root/ -M ncp --hostname=nextcloudpi |& awk "{ print \"${LOG_GUEST} \" \$0 }" &
+#          sleep 60
+#
+#          CONTAINER_CMD=(sudo systemd-run --machine=ncp -P --wait)
+#
+#          success=false
+#          for attempt in {1..30}
+#          do
+#            echo -e "${LOG_CICD} == Wait until container network is available (attempt $attempt/30) =="
+#            ip="$("${CONTAINER_CMD[@]}" bash -c '. /usr/local/etc/library.sh > /dev/null; get_ip')"
+#            [[ -n "$ip" ]] && curl -k "https://$ip/activate/" > /dev/null || { sleep 6; continue; }
+#            success=true
+#            break
+#          done
+#          sudo cat ./raspbian_root/var/log/ncp.log |& awk "{ print \"${LOG_NCP} \" \$0 }"
+#          sudo tail -n 0 -f ./raspbian_root/var/log/ncp.log |& awk "{ print \"${LOG_NCP} \" \$0 }" &
+#
+#          [[ "$success" == "true" ]] || {
+#            echo -e "${LOG_CICD} Could not reach container. Aborting..."
+#            exit 1
+#          }
+#
+#          attempt=0
+#          success=false
+#          for attempt in {1..150}
+#          do
+#            echo -e "${LOG_CICD} Waiting for container startup (attempt $attempt/150)..."
+#            "${CONTAINER_CMD[@]}" journalctl -eu redis.service || true
+#            "${CONTAINER_CMD[@]}" systemctl status php8.1-fpm.service || true
+#            redis_pw="$("${CONTAINER_CMD[@]}" bash -c ". /usr/local/etc/library.sh; get_nc_config_value 'redis\"][\"password'")" \
+#            && redis_socket="$("${CONTAINER_CMD[@]}" bash -c ". /usr/local/etc/library.sh; get_nc_config_value 'redis\"][\"host'")" \
+#            || log_err $? "Error retrieving redis credentials" || true
+#            if { "${CONTAINER_CMD[@]}" -q ncc status |& awk "{ print \"${LOG_DIAG} \" \$0 }" || log_err $? "ncc status check failed"; } \
+#              && { [[ "$("${CONTAINER_CMD[@]}" ncc maintenance:mode)" =~ .*disabled.* ]] || log_err $? "Maintenance mode is enabled or could not be retrieved"; } \
+#              && { "${CONTAINER_CMD[@]}" redis-cli -s "$redis_socket" -a "$redis_pw" set redisready yes |& awk "{ print \"${LOG_DIAG} \" \$0 }" || log_err $? "Failed to set redis variable"; } \
+#              && { "${CONTAINER_CMD[@]}" redis-cli -s "$redis_socket" -a "$redis_pw" get redisready |& awk "{ print \"${LOG_DIAG} \" \$0 }" || log_err $? "Failed to read redis variable"; }
+#            then
+#              echo -e "${LOG_CICD} Startup successful"
+#              success=true
+#              break
+#            fi
+#            attempt=$((attempt + 1))
+#            sleep 5
+#          done
+#
+#          [[ "$success" == "true" ]] || {
+#              echo -e "${LOG_CICD} Timeout reached."
+#              "${CONTAINER_CMD[@]}" -q systemctl status mysql |& awk "{ print \"${LOG_DIAG} \" \$0 }"
+#              "${CONTAINER_CMD[@]}" -q systemctl status redis |& awk "{ print \"${LOG_DIAG} \" \$0 }"
+#              "${CONTAINER_CMD[@]}" -q systemctl status 'php*-fpm' |& awk "{ print \"${LOG_DIAG} \" \$0 }"
+#              "${CONTAINER_CMD[@]}" -q systemctl status apache2 |& awk "{ print \"${LOG_DIAG} \" \$0 }"
+#              "${CONTAINER_CMD[@]}" -q ncp-diag |& awk "{ print \"${LOG_DIAG} \" \$0 }"
+#              exit 1
+#          }
+#
+#          set -x
+#          set +e
+#
+#          success=false
+#          for attempt in {1..5}
+#          do
+#            echo -e "${LOG_CICD} == System Tests (attempt $attempt/5) =="
+#            sudo python tests/system_tests.py --non-interactive --skip-update-test |& awk "{ print \"${LOG_TEST} \" \$0 }"
+#            [[ ${PIPESTATUS[0]} -eq 0 ]] || {
+#              echo -e "${LOG_CICD} System test failed!"
+#              sleep 12
+#              continue
+#            }
+#            success=true
+#            break
+#          done
+#          [[ "$success" == "true" ]] || {
+#            echo -e "${LOG_CICD} System test failed in all attempts!"
+#            exit 1
+#          }
+#
+#          success=false
+#          for attempt in {1..5}
+#          do
+#            echo -e "${LOG_CICD} == Nextcloud Tests (attempt $attempt/5) =="
+#            python tests/nextcloud_tests.py --no-gui --skip-release-check "$ip" 443 4443 |& awk "{ print \"${LOG_TEST} \" \$0 }"
+#            [[ ${PIPESTATUS[0]} -eq 0 ]] || {
+#              echo -e "${LOG_CICD} Nextcloud test failed!"
+#              echo -e "{$LOG_DIAG} Geckodriver logs:"
+#              tail -n 20 geckodriver.log >&2 |& awk "{ print \"${LOG_DIAG} \" \$0 }" || true
+#              echo -e "${LOG_CICD} ================"
+#              echo -e "${LOG_DIAG} ncp.log: "
+#              "${CONTAINER_CMD[@]}" -q ncp /bin/bash -c "tail -n20 /var/log/ncp.log" |& awk "{ print \"${LOG_DIAG} \" \$0 }" || true
+#              echo "================"
+#              echo "${LOG_DIAG} Nextcloud log: "
+#              "${CONTAINER_CMD[@]}" -q ncp /bin/bash -c 'ls -l /opt/ncdata/data/nextcloud.log' |& awk "{ print \"${LOG_DIAG} \" \$0 }" || true
+#              "${CONTAINER_CMD[@]}" -q ncp /bin/bash -c 'cat /opt/ncdata/data/nextcloud.log' |& awk "{ print \"${LOG_DIAG} \" \$0 }" || true
+#              sleep 12
+#              continue
+#            }
+#            success=true
+#            break
+#          done
+#
+#          [[ "$success" == "true" ]] || {
+#            echo -e "${LOG_CICD} Nextcloud test failed in all attempts!"
+#            exit 1
+#          }
+#
+#      - name: Run dist-upgrade
+#        if:  ${{ steps.download-previous-image.outputs.skipped == 'false' }}
+#        run: |
+#          set -ex
+#
+#          source ./library.sh
+#
+#          echo "Updating from $PREVIOUS_VERSION to $VERSION
+#
+#          CONTAINER_CMD=(sudo systemd-run --machine=ncp -P --wait)
+#
+#          sudo grep 'VERSION="11"' ./raspbian_root/etc/os-release || {
+#            echo "Can't dist-upgrade from debian version $(sudo grep 'Version=' ./raspbian_root/etc/os-release)"
+#            exit 1
+#          }
+#          "${CONTAINER_CMD[@]}" DBG=x ncp-dist-upgrade "$VERSION"
+#
+#          sudo machinectl terminate ncp
+#          sudo rm -f ./raspbian_root/opt/ncdata/data/nextcloud.log
+#
+#      - name: Test image after dist-upgrade
+#        if:  ${{ steps.download-previous-image.outputs.skipped == 'false' }}
+#        run: |
+#
+#          log_err() {
+#            rc="${1?}"
+#            msg="${2?}"
+#            echo -e "${LOG_DIAG} $msg" >&2
+#            return $rc
+#          }
+#
+#          sudo systemd-nspawn --boot -D ./raspbian_root/ -M ncp --hostname=nextcloudpi |& awk "{ print \"${LOG_GUEST} \" \$0 }" &
+#          sleep 60
+#
+#          CONTAINER_CMD=(sudo systemd-run --machine=ncp -P --wait)
+#
+#          success=false
+#          for attempt in {1..30}
+#          do
+#            echo -e "${LOG_CICD} == Wait until container network is available (attempt $attempt/30) =="
+#            ip="$("${CONTAINER_CMD[@]}" bash -c '. /usr/local/etc/library.sh > /dev/null; get_ip')"
+#            [[ -n "$ip" ]] && curl -k "https://$ip/activate/" > /dev/null || { sleep 6; continue; }
+#            success=true
+#            break
+#          done
+#          sudo cat ./raspbian_root/var/log/ncp.log |& awk "{ print \"${LOG_NCP} \" \$0 }"
+#          sudo tail -n 0 -f ./raspbian_root/var/log/ncp.log |& awk "{ print \"${LOG_NCP} \" \$0 }" &
+#
+#          [[ "$success" == "true" ]] || {
+#            echo -e "${LOG_CICD} Could not reach container. Aborting..."
+#            exit 1
+#          }
+#
+#          attempt=0
+#          success=false
+#          for attempt in {1..150}
+#          do
+#            echo -e "${LOG_CICD} Waiting for container startup (attempt $attempt/150)..."
+#            "${CONTAINER_CMD[@]}" journalctl -eu redis.service || true
+#            "${CONTAINER_CMD[@]}" systemctl status php8.1-fpm.service || true
+#            redis_pw="$("${CONTAINER_CMD[@]}" bash -c ". /usr/local/etc/library.sh; get_nc_config_value 'redis\"][\"password'")" \
+#            && redis_socket="$("${CONTAINER_CMD[@]}" bash -c ". /usr/local/etc/library.sh; get_nc_config_value 'redis\"][\"host'")" \
+#            || log_err $? "Error retrieving redis credentials" || true
+#            if { "${CONTAINER_CMD[@]}" -q ncc status |& awk "{ print \"${LOG_DIAG} \" \$0 }" || log_err $? "ncc status check failed"; } \
+#              && { [[ "$("${CONTAINER_CMD[@]}" ncc maintenance:mode)" =~ .*disabled.* ]] || log_err $? "Maintenance mode is enabled or could not be retrieved"; } \
+#              && { "${CONTAINER_CMD[@]}" redis-cli -s "$redis_socket" -a "$redis_pw" set redisready yes |& awk "{ print \"${LOG_DIAG} \" \$0 }" || log_err $? "Failed to set redis variable"; } \
+#              && { "${CONTAINER_CMD[@]}" redis-cli -s "$redis_socket" -a "$redis_pw" get redisready |& awk "{ print \"${LOG_DIAG} \" \$0 }" || log_err $? "Failed to read redis variable"; }
+#            then
+#              echo -e "${LOG_CICD} Startup successful"
+#              success=true
+#              break
+#            fi
+#            attempt=$((attempt + 1))
+#            sleep 5
+#          done
+#
+#          [[ "$success" == "true" ]] || {
+#              echo -e "${LOG_CICD} Timeout reached."
+#              "${CONTAINER_CMD[@]}" -q systemctl status mysql |& awk "{ print \"${LOG_DIAG} \" \$0 }"
+#              "${CONTAINER_CMD[@]}" -q systemctl status redis |& awk "{ print \"${LOG_DIAG} \" \$0 }"
+#              "${CONTAINER_CMD[@]}" -q systemctl status 'php*-fpm' |& awk "{ print \"${LOG_DIAG} \" \$0 }"
+#              "${CONTAINER_CMD[@]}" -q systemctl status apache2 |& awk "{ print \"${LOG_DIAG} \" \$0 }"
+#              "${CONTAINER_CMD[@]}" -q ncp-diag |& awk "{ print \"${LOG_DIAG} \" \$0 }"
+#              exit 1
+#          }
+#
+#          set -x
+#          set +e
+#
+#          success=false
+#          for attempt in {1..5}
+#          do
+#            echo -e "${LOG_CICD} == Activation Tests (attempt $attempt/5) =="
+#            python tests/activation_tests.py -t 300 --no-gui "$ip" 443 4443 |& awk "{ print \"${LOG_TEST} \" \$0 }"
+#            [[ ${PIPESTATUS[0]} -eq 0 ]] || {
+#              echo -e "${LOG_CICD} Activation test failed!"
+#              echo -e "${LOG_DIAG} Geckodriver logs:"
+#              tail -n 20 geckodriver.log >&2 |& awk "{ print \"${LOG_DIAG} \" \$0 }" || true
+#              echo "================"
+#              echo -e "${LOG_DIAG} mysql: "
+#              "${CONTAINER_CMD[@]}" -q ncp-diag |& awk "{ print \"${LOG_DIAG} \" \$0 }" || true
+#              "${CONTAINER_CMD[@]}" -q systemctl status mysql |& awk "{ print \"${LOG_DIAG} \" \$0 }" || true
+#              sleep 12
+#              continue
+#            }
+#            success=true
+#            break
+#          done
+#          [[ "$success" == "true" ]] || {
+#            echo -e "${LOG_CICD} Activation test failed in all attempts!"
+#            exit 1
+#          }
+#
+#          success=false
+#          for attempt in {1..5}
+#          do
+#            echo -e "${LOG_CICD} == System Tests (attempt $attempt/5) =="
+#            sudo python tests/system_tests.py --non-interactive |& awk "{ print \"${LOG_TEST} \" \$0 }"
+#            [[ ${PIPESTATUS[0]} -eq 0 ]] || {
+#              echo -e "${LOG_CICD} System test failed!"
+#              sleep 12
+#              continue
+#            }
+#            success=true
+#            break
+#          done
+#          [[ "$success" == "true" ]] || {
+#            echo -e "${LOG_CICD} System test failed in all attempts!"
+#            exit 1
+#          }
+#
+#          success=false
+#          for attempt in {1..5}
+#          do
+#            echo -e "${LOG_CICD} == Nextcloud Tests (attempt $attempt/5) =="
+#            python tests/nextcloud_tests.py --no-gui "$ip" 443 4443 |& awk "{ print \"${LOG_TEST} \" \$0 }"
+#            [[ ${PIPESTATUS[0]} -eq 0 ]] || {
+#              echo -e "${LOG_CICD} Nextcloud test failed!"
+#              echo -e "{$LOG_DIAG} Geckodriver logs:"
+#              tail -n 20 geckodriver.log >&2 |& awk "{ print \"${LOG_DIAG} \" \$0 }" || true
+#              echo -e "${LOG_CICD} ================"
+#              echo -e "${LOG_DIAG} ncp.log: "
+#              "${CONTAINER_CMD[@]}" -q ncp /bin/bash -c "tail -n20 /var/log/ncp.log" |& awk "{ print \"${LOG_DIAG} \" \$0 }" || true
+#              echo "================"
+#              echo "${LOG_DIAG} Nextcloud log: "
+#              "${CONTAINER_CMD[@]}" -q ncp /bin/bash -c 'ls -l /opt/ncdata/data/nextcloud.log' |& awk "{ print \"${LOG_DIAG} \" \$0 }" || true
+#              "${CONTAINER_CMD[@]}" -q ncp /bin/bash -c 'cat /opt/ncdata/data/nextcloud.log' |& awk "{ print \"${LOG_DIAG} \" \$0 }" || true
+#              sleep 12
+#              continue
+#            }
+#            success=true
+#            break
+#          done
+#
+#          [[ "$success" == "true" ]] || {
+#            echo -e "${LOG_CICD} Nextcloud test failed in all attempts!"
+#            exit 1
+#          }

.github/workflows/lxd-trigger.yml

@@ -14,4 +14,4 @@ jobs:
   lxd:
     uses: ./.github/workflows/build-lxd.yml
     with:
-      git_ref: "${{ github.ref }}"
+      git_ref: "${{ github.head_ref || github.ref_name }}"

.github/workflows/publish-image.yml

@@ -32,7 +32,7 @@ jobs:
         with:
           ref: "${{ env.VERSION }}"
       - name: "Download artifact"
-        uses: actions/download-artifact@v3
+        uses: actions/download-artifact@v4
         with:
           name: ${{ inputs.artifact_id }}
           path: artifacts
@@ -43,7 +43,7 @@ jobs:
         run: |
           set -ex
           mkdir -p publish
-          mv artifacts/${{ inputs.artifact_file }} publish/
+          mv "artifacts/${{ inputs.artifact_file }}" publish/
           cd publish
           
           asset="${IMG}"
@@ -64,7 +64,7 @@ jobs:
           success=false
           for i in {1..5}
           do
-            body="$(hub release show -f "%b" "${VERSION}")"
+            body="$(gh release view --json body "${VERSION}" | jq -r '.body')"
             if ! [[ "$body" =~ .*'**Checksums:**'.* ]]
             then
             
@@ -80,7 +80,7 @@ jobs:
           \`\`\`"
           
             gh release edit "${VERSION?}" -n "$body"
-            if hub release show -f "%b" "${VERSION}" | grep "$checksum"
+            if gh release view --json body "${VERSION}" | jq -r '.body' | grep "$checksum"
             then
               success=true
               break

.github/workflows/release.yml

@@ -16,11 +16,6 @@ on:
         description: 'Build/test LXD image?'
         required: false
         default: true
-      docker:
-        type: boolean
-        description: 'Build/test docker images?'
-        required: false
-        default: true
       installer:
         type: boolean
         description: 'Run curl-installer/upgrade tests?'
@@ -28,16 +23,19 @@ on:
         default: true
       release:
         type: boolean
-        description: 'Release images to GH and docker registry'
+        description: 'Release images to GH'
         required: false
         default: false
+      test_image_url:
+        type: string
+        description: 'URL for test image to test instead of actual armbian builds'
+        required: false
+        default: ''
   push:
     tags:
       - "v*"
-      - "docker-v*"
     branches:
       - master
-      - docker-stable
 
 permissions:
   contents: write
@@ -58,30 +56,6 @@ jobs:
       arch: 'arm64'
     secrets: inherit
 
-  docker-x86:
-    if: ${{ inputs.docker || ( github.event_name != 'workflow_dispatch' && startsWith(github.ref_name, 'docker-') ) }}
-    uses: ./.github/workflows/build-docker.yml
-    with:
-      git_ref: "${{ inputs.git_ref || github.head_ref || github.ref_name }}"
-      arch: "x86"
-    secrets: inherit
-
-  docker-armhf:
-    if: ${{ inputs.docker || ( github.event_name != 'workflow_dispatch' && startsWith(github.ref_name, 'docker-') ) }}
-    uses: ./.github/workflows/build-docker.yml
-    with:
-      git_ref: "${{ inputs.git_ref || github.head_ref || github.ref_name }}"
-      arch: "armhf"
-    secrets: inherit
-
-  docker-arm64:
-    if: ${{ inputs.docker || ( github.event_name != 'workflow_dispatch' && startsWith(github.ref_name, 'docker-') ) }}
-    uses: ./.github/workflows/build-docker.yml
-    with:
-      git_ref: "${{ inputs.git_ref || github.head_ref || github.ref_name }}"
-      arch: "arm64"
-    secrets: inherit
-
   test-curl-installer:
     if: ${{ inputs.installer || ( github.event_name != 'workflow_dispatch' && !startsWith(github.ref_name, 'docker-') ) }}
     uses: ./.github/workflows/vm-tests.yml
@@ -94,13 +68,23 @@ jobs:
     uses: ./.github/workflows/build-sd-images.yml
     with:
       git_ref: "${{ inputs.git_ref || github.head_ref || github.ref_name }}"
-      board_id: raspberrypi
-      board_name: RaspberryPi
+      board_id: rpi4b
+      board_name: RaspberryPi 4+
+    secrets: inherit
+
+  armbian-test-image:
+    if: ${{ inputs.test_image_url != '' }}
+    uses: ./.github/workflows/build-sd-images.yml
+    with:
+      git_ref: "${{ inputs.git_ref || github.head_ref || github.ref_name }}"
+      board_id: testimage
+      board_name: TestImage
+      test_image_url: "${{ inputs.test_image_url }}"
     secrets: inherit
 
   # TODO: Fix 32bit armbian images
   odroidxu4:
-    # if: ${{ inputs.sd-images || github.event_name != 'workflow_dispatch' }}
+    # if: ${{ inputs.sd-images || ( github.event_name != 'workflow_dispatch' && !startsWith(github.ref_name, 'docker-') ) }}
     if: ${{ false }}
     uses: ./.github/workflows/build-sd-images.yml
     with:
@@ -126,7 +110,7 @@ jobs:
     secrets: inherit
   # TODO: Fix 32bit armbian images
   bananapi:
-    #if: ${{ inputs.sd-images || github.event_name != 'workflow_dispatch' }}
+    # if: ${{ inputs.sd-images || ( github.event_name != 'workflow_dispatch' && !startsWith(github.ref_name, 'docker-') ) }}
     if: ${{ false }}
     uses: ./.github/workflows/build-sd-images.yml
     with:
@@ -158,6 +142,22 @@ jobs:
       board_id: odroidc2
       board_name: OdroidC2
     secrets: inherit
+  orangepi5:
+    if: ${{ inputs.sd-images || ( github.event_name != 'workflow_dispatch' && !startsWith(github.ref_name, 'docker-') ) }}
+    uses: ./.github/workflows/build-sd-images.yml
+    with:
+      git_ref: "${{ inputs.git_ref || github.head_ref || github.ref_name }}"
+      board_id: orangepi5
+      board_name: OrangePi5
+    secrets: inherit
+  orangepi5plus:
+    if: ${{ inputs.sd-images || ( github.event_name != 'workflow_dispatch' && !startsWith(github.ref_name, 'docker-') ) }}
+    uses: ./.github/workflows/build-sd-images.yml
+    with:
+      git_ref: "${{ inputs.git_ref || github.head_ref || github.ref_name }}"
+      board_id: orangepi5-plus
+      board_name: OrangePi5Plus
+    secrets: inherit
 
   lxd-x86-release:
     needs:
@@ -304,55 +304,30 @@ jobs:
       artifact_file: "${{ needs.odroidc2.outputs.artifact_file }}"
       dry_run: ${{ (!inputs.release && github.event_name == 'workflow_dispatch') || github.ref_type != 'tag' || !(github.ref_protected || startsWith(github.ref, 'refs/tags/v'))  }}
     secrets: inherit
-
-  docker-release:
+  orangepi5-release:
     needs:
-      - docker-x86
-      - docker-armhf
-      - docker-arm64
-    if: ${{ (inputs.release || github.event_name != 'workflow_dispatch') && github.ref_type == 'tag' && (github.ref_protected || startsWith(github.ref, 'refs/tags/docker-v'))  }}
-    runs-on: ubuntu-latest
-    steps:
-      - name: Login to DockerHub
-        uses: docker/login-action@v1
-        with:
-          username: ${{ secrets.DOCKERHUB_USERNAME }}
-          password: ${{ secrets.DOCKERHUB_TOKEN }}
-
-      - name: Checkout code
-        uses: actions/checkout@v3
-
-      - name: Create manifest and push as tag to docker hub
-        run: |
-          . ./build/buildlib.sh
-          version="${version#docker-}"
-          
-          for arch in x86 armhf arm64
-          do
-            docker pull "thecalcaholic/ncp-internal-${arch}:${{ github.run_id }}"
-            docker tag "thecalcaholic/ncp-internal-${arch}:${{ github.run_id }}" "ownyourbits/nextcloudpi-${arch}:${version?}"
-            docker tag "ownyourbits/nextcloudpi-${arch}:${version?}" "ownyourbits/nextcloudpi-${arch}:latest"
-            docker push "ownyourbits/nextcloudpi-${arch}:${version?}"
-            [[ "$version" =~ ^(docker-)?v[0-9]+'.'[0-9]+'.'[0-9]+$ ]] && docker push "ownyourbits/nextcloudpi-${arch}:latest"
-          done
-
-          docker manifest create ownyourbits/nextcloudpi:${version?} \
-            ownyourbits/nextcloudpi-armhf:${version?} \
-            ownyourbits/nextcloudpi-x86:${version?} \
-            ownyourbits/nextcloudpi-arm64:${version?}
-          docker manifest push ownyourbits/nextcloudpi:${version?}
-
-      - name: Create manifest and push as latest to docker hub
-        run: |
-          [[ "${{ github.ref }}" =~ ^'refs/tags/'(docker-)?'v'[0-9]+'.'[0-9]+'.'[0-9]+$ ]] || {
-            echo "not tagging latest - is a pre-/beta-release"
-            exit 0
-          }
-          docker manifest create ownyourbits/nextcloudpi:latest \
-          ownyourbits/nextcloudpi-armhf:latest \
-          ownyourbits/nextcloudpi-x86:latest \
-          ownyourbits/nextcloudpi-arm64:latest
-          docker manifest push ownyourbits/nextcloudpi:latest
+      - orangepi5
+      - github-release
+    if: ${{ inputs.sd-images || github.event_name != 'workflow_dispatch' }}
+    uses: ./.github/workflows/publish-image.yml
+    with:
+      git_ref: "${{ inputs.git_ref || github.head_ref || github.ref_name }}"
+      artifact_id: "${{ needs.orangepi5.outputs.artifact_name }}"
+      artifact_file: "${{ needs.orangepi5.outputs.artifact_file }}"
+      dry_run: ${{ (!inputs.release && github.event_name == 'workflow_dispatch') || github.ref_type != 'tag' || !(github.ref_protected || startsWith(github.ref, 'refs/tags/v'))  }}
+    secrets: inherit
+  orangepi5plus-release:
+    needs:
+      - orangepi5plus
+      - github-release
+    if: ${{ inputs.sd-images || github.event_name != 'workflow_dispatch' }}
+    uses: ./.github/workflows/publish-image.yml
+    with:
+      git_ref: "${{ inputs.git_ref || github.head_ref || github.ref_name }}"
+      artifact_id: "${{ needs.orangepi5plus.outputs.artifact_name }}"
+      artifact_file: "${{ needs.orangepi5plus.outputs.artifact_file }}"
+      dry_run: ${{ (!inputs.release && github.event_name == 'workflow_dispatch') || github.ref_type != 'tag' || !(github.ref_protected || startsWith(github.ref, 'refs/tags/v'))  }}
+    secrets: inherit
 
   github-release:
     if: ${{ github.event_name == 'workflow_dispatch' || !startsWith(github.ref_name, 'docker-') }}
@@ -383,16 +358,28 @@ jobs:
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
         run: |
+          
+          NC_VERSION="$(jq -r '.nextcloud_version' < etc/ncp.cfg)"
+          PHP_VERSION="$(jq -r '.php_version' < etc/ncp.cfg)"
+          DEBIAN_VERSION="$(jq -r '.release' < etc/ncp.cfg)"
+          ARMBIAN_VERSIOn="$(cat build/armbian/armbian_version)"
+          
           subject="$(git tag -n10 --format="%(contents:subject)" "${{ env.VERSION }}")"
-          body="$(git tag -n10 --format="%(contents:body)" "${{ env.VERSION }}")"
+          body="$(git tag -n30 --format="%(contents:body)" "${{ env.VERSION }}")"
           separator="
 
           ---
           
           "
 
-          hub release create --draft -F - "${{ env.VERSION }}" <<EOF
+          gh release create --draft -F - "${{ env.VERSION }}" <<EOF
           ${subject:-No message found}
+          
+          ### Included Software
+          Nextcloud ${NC_VERSION} (can be automatically updated to latest minor version) 
+          PHP ${PHP_VERSION}
+          Debian ${DEBIAN_VERSION}
+          Armbian ${ARMBIAN_VERSION}
 
           ${body:+${body}${separator}}
 

.github/workflows/vm-tests.yml

@@ -18,126 +18,321 @@ on:
       - master
 
 jobs:
-  setup-installation-test-instance:
+  installation-test:
     runs-on: ubuntu-latest
+    container:
+      image: thecalcaholic/ncp-test-automation:bookworm
+      env:
+        HCLOUD_TOKEN: "${{ secrets.TEST_AUTOMATION_HCLOUD_API_TOKEN }}"
+        UID: "${{ github.run_id }}-install"
     outputs:
       server_address: ${{ steps.create-test-instance.outputs.server_address }}
       snapshot_id: ${{ steps.create-test-instance.outputs.snapshot_id }}
       test_server_id: ${{ steps.create-test-instance.outputs.test_server_id }}
       version: ${{ env.VERSION }}
+      test_result: ${{ steps.final_test.outputs.test_result }}
+      ssh_artifact_name: ${{ env.SSH_ARTIFACT_NAME }}
     env:
       VERSION: "${{ inputs.git_ref || github.head_ref || github.ref_name }}"
+      HOME: /root
+      SSH_ARTIFACT_NAME: "${{ github.run_id }}-install-ssh"
+      UID: "${{ github.run_id }}-install"
+    defaults:
+      run:
+        shell: bash
     steps:
       - uses: actions/checkout@v3
-      - run: |
-          set -e
-          mkdir -p ./.ssh
-          ssh-keygen -t ed25519 -f ".ssh/automation_ssh_key"
-      - name: upload ssh private key to artifact store
-        uses: actions/upload-artifact@v3
         with:
-          name: ${{ github.run_id }}-install-ssh-privkey
-          path: .ssh
+          path: /__w/nextcloudpi/nextcloudpi
+      - name: Generate ssh keypair
+        working-directory: /__w/nextcloudpi/nextcloudpi
+        run: |
+          set -e
+          mkdir -p .ssh
+          ssh-keygen -t ed25519 -f ".ssh/automation_ssh_key"
+          . /ncp-test-automation/bin/entrypoint.sh
+      - name: upload ssh private key to artifact store
+        uses: actions/upload-artifact@v4
+        with:
+          name: "${{ env.SSH_ARTIFACT_NAME }}"
+          path: /__w/nextcloudpi/nextcloudpi/.ssh
           if-no-files-found: error
+          include-hidden-files: true
       - id: create-test-instance
         uses: ./.github/actions/create-test-instance
         with:
           version: ${{ env.VERSION }}
-          uid: "${{ github.run_id }}-install"
+          uid: "${{ env.UID }}"
           hcloud_token: ${{ secrets.TEST_AUTOMATION_HCLOUD_API_TOKEN }}
-          server_type: "cx11"
+          server_type: "cx22"
+      - name: set instance variables
+        run: |
+          echo "SERVER_ADDRESS=${{ steps.create-test-instance.outputs.server_address }}" >> "$GITHUB_ENV"
+          echo "SNAPSHOT_ID=${{ steps.create-test-instance.outputs.snapshot_id }}" >> "$GITHUB_ENV"
+      - name: Test postinstall VM
+        id: final_test
+        working-directory: /ncp-test-automation/bin
+        run: |
+          set -e
+          echo "Setup ssh"
+          chmod 0600 /__w/nextcloudpi/nextcloudpi/.ssh/automation_ssh_key
+          eval "$(ssh-agent)"
+          ssh-add /__w/nextcloudpi/nextcloudpi/.ssh/automation_ssh_key
+          
+          source ./library.sh
+          
+          trap '[ $? -eq 0 ] || echo "test_result=failure" >> "$GITHUB_OUTPUT"; terminate-ssh-port-forwarding "${SERVER_ADDRESS}"' EXIT 1 2
+          
+          setup-ssh-port-forwarding "$SERVER_ADDRESS"
+          
+          echo "Run integration tests"
+          ssh -o "StrictHostKeyChecking=no" -o "UserKnownHostsFile=/dev/null" "root@${SERVER_ADDRESS}" cat /usr/local/etc/instance.cfg
+          set -x
+          test-ncp-instance -a -f "$SNAPSHOT_ID" -b "${VERSION}" --systemtest-args "--skip-update-test" "root@${SERVER_ADDRESS}" "localhost" "8443" "9443" || {
+          
+            echo "Integration tests failed"
+            echo "Here are the last lines of ncp-install.log:"
+            echo "==========================================="
+            ssh "${SSH_OPTIONS[@]}" "root@${SERVER_ADDRESS}" tail /var/log/ncp-install.log;
+            echo "==========================================="
+            echo "and ncp.log:"
+            echo "==========================================="
+            ssh "${SSH_OPTIONS[@]}" "root@${SERVER_ADDRESS}" tail /var/log/ncp.log;
+            echo "==========================================="
+            exit 1
+          }
+          echo "test_result=success" >> "$GITHUB_OUTPUT";
 
-  setup-update-test-instance:
+#  update-test:
+#    runs-on: ubuntu-latest
+#    container:
+#      image: thecalcaholic/ncp-test-automation:bullseye
+#      env:
+#        HCLOUD_TOKEN: "${{ secrets.TEST_AUTOMATION_HCLOUD_API_TOKEN }}"
+#        UID: "${{ github.run_id }}-update"
+#    defaults:
+#      run:
+#        shell: bash
+#    outputs:
+#      server_address: ${{ steps.create-test-instance.outputs.server_address }}
+#      snapshot_id: ${{ steps.create-test-instance.outputs.snapshot_id }}
+#      test_server_id: ${{ steps.create-test-instance.outputs.test_server_id }}
+#      previous_version: ${{ steps.find-version.outputs.previous_version }}
+#      version: ${{ env.VERSION }}
+#      test_result: ${{ steps.final_test.outputs.test_result }}
+#      ssh_artifact_name: ${{ env.SSH_ARTIFACT_NAME }}
+#    env:
+#      VERSION: "${{ inputs.git_ref || github.head_ref || github.ref_name }}"
+#      HOME: /root
+#      SSH_ARTIFACT_NAME: "${{ github.run_id }}-update-ssh"
+#      UID: "${{ github.run_id }}-update"
+#    steps:
+#      - uses: actions/checkout@v3
+#        with:
+#          fetch-depth: 0
+#          path: /__w/nextcloudpi/nextcloudpi
+#      - name: find reference version
+#        working-directory: /__w/nextcloudpi/nextcloudpi
+#        shell: bash
+#        id: find-version
+#        run: |
+#          chown -R "$(id -u):$(id -g)" .
+#          set -e
+#          if [[ -n "${{ github.base_ref }}" ]]
+#          then
+#            version="${{ github.base_ref }}"
+#          elif [[ "${{ github.ref }}" == "refs/heads/devel" ]]
+#          then
+#            version="master"
+#          else
+#            git fetch -fu --tags origin ${{ github.ref }}:${{ github.ref }}
+#            version="$(git describe --tags)"
+#            [[ "$version" =~ .*-.*-.* ]] || {
+#              git checkout HEAD~1
+#              version="$(git describe --tags)"
+#            }
+#            version="${version%-*-*}"
+#          fi
+#          echo "Previous version is '$version'"
+#          #TODO: Revert to dynamically found version
+#          #echo "PREVIOUS_VERSION=${version}" >> "$GITHUB_ENV"
+#          echo "PREVIOUS_VERSION=v1.53.3" >> "$GITHUB_ENV"
+#      - name: Generate ssh key
+#        run: |
+#          set -x
+#          mkdir -p /__w/nextcloudpi/nextcloudpi/.ssh
+#          ssh-keygen -t ed25519 -f "/__w/nextcloudpi/nextcloudpi/.ssh/automation_ssh_key"
+#          . /ncp-test-automation/bin/entrypoint.sh
+#      - name: upload ssh private key to artifact store
+#        uses: actions/upload-artifact@v3
+#        with:
+#          name: "${{ env.SSH_ARTIFACT_NAME }}"
+#          path: /__w/nextcloudpi/nextcloudpi/.ssh
+#          if-no-files-found: error
+#      - id: create-test-instance
+#        uses: ./.github/actions/create-test-instance-bullseye
+#        with:
+#          version: "${{ env.PREVIOUS_VERSION }}"
+#          uid: "${{ env.UID }}"
+#          hcloud_token: ${{ secrets.TEST_AUTOMATION_HCLOUD_API_TOKEN }}
+#          server_type: "cx11"
+#      - name: Set instance variables
+#        run: |
+#          echo "SERVER_ADDRESS=${{ steps.create-test-instance.outputs.server_address }}" >> "$GITHUB_ENV"
+#          echo "SNAPSHOT_ID=${{ steps.create-test-instance.outputs.snapshot_id }}" >> "$GITHUB_ENV"
+#      - uses: actions/checkout@v3
+#        with:
+#          repository: 'theCalcaholic/ncp-test-automation'
+#          ref: "bullseye"
+#          path: /__w/nextcloudpi/nextcloudpi/ncp-test-automation
+#      - name: Activate and Test postinstall VM
+#        working-directory: /__w/nextcloudpi/nextcloudpi/ncp-test-automation/bin
+#        run: |
+#          set -e
+#          echo "Setup ssh"
+#          chmod 0600 /__w/nextcloudpi/nextcloudpi/.ssh/automation_ssh_key
+#          eval "$(ssh-agent)"
+#          ssh-add /__w/nextcloudpi/nextcloudpi/.ssh/automation_ssh_key
+#
+#          source ./library.sh
+#
+#          trap 'terminate-ssh-port-forwarding "${SERVER_ADDRESS}"' EXIT 1 2
+#
+#          setup-ssh-port-forwarding "$SERVER_ADDRESS"
+#
+#          echo "Run integration tests"
+#          ssh -o "StrictHostKeyChecking=no" -o "UserKnownHostsFile=/dev/null" "root@${SERVER_ADDRESS}" cat /usr/local/etc/instance.cfg
+#          test-ncp-instance -a -f "$SNAPSHOT_ID" -b "${VERSION}" --systemtest-args "--skip-update-test" --nc-test-args "--skip-release-check" "root@${SERVER_ADDRESS}" "localhost" "8443" "9443" || {
+#
+#            echo "Integration tests failed"
+#            echo "Here are the last lines of ncp-install.log:"
+#            echo "==========================================="
+#            ssh "${SSH_OPTIONS[@]}" "root@${SERVER_ADDRESS}" tail /var/log/ncp-install.log;
+#            echo "==========================================="
+#            echo "and ncp.log:"
+#            echo "==========================================="
+#            ssh "${SSH_OPTIONS[@]}" "root@${SERVER_ADDRESS}" tail /var/log/ncp.log;
+#            echo "==========================================="
+#            exit 1
+#          }
+#      - name: perform update
+#        working-directory: /__w/nextcloudpi/nextcloudpi/ncp-test-automation/bin
+#        run: |
+#          set -e
+#
+#          echo "Setup ssh"
+#          chmod 0600 /__w/nextcloudpi/nextcloudpi/.ssh/automation_ssh_key
+#          eval "$(ssh-agent)"
+#          ssh-add /__w/nextcloudpi/nextcloudpi/.ssh/automation_ssh_key
+#
+#          source ./library.sh
+#
+#          echo "Updating from $PREVIOUS_VERSION to $VERSION"
+#          ssh-keygen -f "$HOME/.ssh/known_hosts" -R "${SERVER_ADDRESS}" 2> /dev/null || true
+#          ssh "${SSH_OPTIONS[@]}" "root@${SERVER_ADDRESS}" "ncp-update '$VERSION'"
+#      - name: Run integration tests after update
+#        id: final_test
+#        working-directory: /__w/nextcloudpi/nextcloudpi/ncp-test-automation/bin
+#        run: |
+#          set -e
+#
+#          echo "Setup ssh"
+#          eval "$(ssh-agent)"
+#          ssh-add /__w/nextcloudpi/nextcloudpi/.ssh/automation_ssh_key
+#
+#          source ./library.sh
+#
+#          trap '[ $? -eq 0 ] || echo "test_result=failure" >> "$GITHUB_OUTPUT"; terminate-ssh-port-forwarding "${SERVER_ADDRESS}"' EXIT 1 2
+#
+#          echo "Run integration tests"
+#          setup-ssh-port-forwarding "$SERVER_ADDRESS"
+#          NC_TEST_ARGS=()
+#          ssh "${SSH_OPTIONS[@]}" "root@${SERVER_ADDRESS}" cat /etc/os-release | grep VERSION_ID=12 || NC_TEST_ARGS+=("--skip-release-check")
+#          set -x
+#          test-ncp-instance -f "$SNAPSHOT_ID" -b "${VERSION}" --nc-test-args "$NC_TEST_ARGS" --systemtest-args "--skip-update-test" "root@${SERVER_ADDRESS}" "localhost" "8443" "9443" || {
+#
+#            echo "Integration tests failed"
+#            echo "Here are the last lines of ncp-install.log:"
+#            echo "==========================================="
+#            ssh "${SSH_OPTIONS[@]}" "root@${SERVER_ADDRESS}" tail /var/log/ncp-install.log;
+#            echo "==========================================="
+#            echo "and ncp.log:"
+#            echo "==========================================="
+#            ssh "${SSH_OPTIONS[@]}" "root@${SERVER_ADDRESS}" tail /var/log/ncp.log;
+#            echo "==========================================="
+#            exit 1
+#          }
+#
+#          echo "test_result=success" >> "$GITHUB_OUTPUT";
+
+  dist-upgrade-test:
     runs-on: ubuntu-latest
+    container:
+      image: thecalcaholic/ncp-test-automation:bullseye
+      env:
+        HCLOUD_TOKEN: "${{ secrets.TEST_AUTOMATION_HCLOUD_API_TOKEN }}"
+        UID: "${{ github.run_id }}-distupgrade"
+    defaults:
+      run:
+        shell: bash
     outputs:
+      test_result: ${{ steps.final_test.outputs.test_result }}
+      ssh_artifact_name: ${{ env.SSH_ARTIFACT_NAME }}
       server_address: ${{ steps.create-test-instance.outputs.server_address }}
       snapshot_id: ${{ steps.create-test-instance.outputs.snapshot_id }}
       test_server_id: ${{ steps.create-test-instance.outputs.test_server_id }}
-      previous_version: ${{ steps.find-version.outputs.previous_version }}
+      previous_version: ${{ env.PREVIOUS_VERSION }}
       version: ${{ env.VERSION }}
     env:
+      PREVIOUS_VERSION: "v1.53.3"
       VERSION: "${{ inputs.git_ref || github.head_ref || github.ref_name }}"
+      HOME: /root
+      SSH_ARTIFACT_NAME: "${{ github.run_id }}-distupgrade-ssh"
+      UID: "${{ github.run_id }}-distupgrade"
     steps:
       - uses: actions/checkout@v3
         with:
           fetch-depth: 0
-      - name: find reference version
-        shell: bash
-        id: find-version
-        run: |
-          set -e
-          if [[ -n "${{ github.base_ref }}" ]]
-          then
-            version="${{ github.base_ref }}"
-          elif [[ "${{ github.ref }}" == "refs/heads/devel" ]]
-          then
-            version="master"
-          else
-            git fetch -fu --tags origin ${{ github.ref }}:${{ github.ref }}
-            version="$(git describe --tags)"
-            [[ "$version" =~ .*-.*-.* ]] || {
-              git checkout HEAD~1
-              version="$(git describe --tags)"
-            }
-            version="${version%-*-*}"
-          fi
-          echo "Previous version is '$version'"
-          echo "previous_version=${version}" >> $GITHUB_OUTPUT
+          path: /__w/nextcloudpi/nextcloudpi
       - name: Generate ssh key
         run: |
           set -x
-          mkdir -p ./.ssh
-          ssh-keygen -t ed25519 -f ".ssh/automation_ssh_key"
+          chown -R "$(id -u):$(id -g)" /__w/nextcloudpi/nextcloudpi
+          mkdir -p /__w/nextcloudpi/nextcloudpi/.ssh
+          ssh-keygen -t ed25519 -f "/__w/nextcloudpi/nextcloudpi/.ssh/automation_ssh_key"
+          . /ncp-test-automation/bin/entrypoint.sh
       - name: upload ssh private key to artifact store
-        uses: actions/upload-artifact@v3
+        uses: actions/upload-artifact@v4
         with:
-          name: ${{ github.run_id }}-update-ssh-privkey
-          path: .ssh
+          name: "${{ env.SSH_ARTIFACT_NAME }}"
+          path: /__w/nextcloudpi/nextcloudpi/.ssh
           if-no-files-found: error
+          include-hidden-files: true
       - id: create-test-instance
-        uses: ./.github/actions/create-test-instance
+        uses: ./.github/actions/create-test-instance-bullseye
         with:
-          version: "${{ steps.find-version.outputs.previous_version }}"
-          uid: "${{ github.run_id }}-update"
+          version: "${{ env.PREVIOUS_VERSION }}"
+          uid: "${{ env.UID }}"
           hcloud_token: ${{ secrets.TEST_AUTOMATION_HCLOUD_API_TOKEN }}
-          server_type: "cx11"
-
-  run-installation-test:
-    needs:
-      - setup-installation-test-instance
-    runs-on: ubuntu-latest
-
-    container:
-      image: thecalcaholic/ncp-test-automation:latest
-      env:
-        HCLOUD_TOKEN: "${{ secrets.TEST_AUTOMATION_HCLOUD_API_TOKEN }}"
-        UID: "${{ github.run_id }}-install"
-    env:
-      VERSION: ${{ needs.setup-installation-test-instance.outputs.version }}
-      SERVER_ADDRESS: "${{ needs.setup-installation-test-instance.outputs.server_address }}"
-      SNAPSHOT_ID: "${{ needs.setup-installation-test-instance.outputs.snapshot_id }}"
-      HOME: /root
-    defaults:
-      run:
-        shell: bash
-    steps:
+          server_type: "cx22"
+      - name: Set instance variables
+        run: |
+          echo "SERVER_ADDRESS=${{ steps.create-test-instance.outputs.server_address }}" >> "$GITHUB_ENV"
+          echo "SNAPSHOT_ID=${{ steps.create-test-instance.outputs.snapshot_id }}" >> "$GITHUB_ENV"
       - uses: actions/checkout@v3
         with:
           repository: 'theCalcaholic/ncp-test-automation'
-      - name: download ssh private key from artifact store
-        uses: actions/download-artifact@v3
-        with:
-          name: ${{ github.run_id }}-install-ssh-privkey
-          path: .ssh
-      - name: Test postinstall VM
+          ref: "bullseye"
+          path: /__w/nextcloudpi/nextcloudpi/ncp-test-automation
+      - name: Activate and Test postinstall VM
+        working-directory: /__w/nextcloudpi/nextcloudpi/ncp-test-automation/bin
         run: |
           set -e
           echo "Setup ssh"
-          chmod 0600 ./.ssh/automation_ssh_key
+          chmod 0600 /__w/nextcloudpi/nextcloudpi/.ssh/automation_ssh_key
           eval "$(ssh-agent)"
-          ssh-add ./.ssh/automation_ssh_key
+          ssh-add /__w/nextcloudpi/nextcloudpi/.ssh/automation_ssh_key
           
-          cd bin
           source ./library.sh
           
           trap 'terminate-ssh-port-forwarding "${SERVER_ADDRESS}"' EXIT 1 2
@@ -145,8 +340,9 @@ jobs:
           setup-ssh-port-forwarding "$SERVER_ADDRESS"
           
           echo "Run integration tests"
-          test-ncp-instance -a -f "$SNAPSHOT_ID" -b "${VERSION}" "root@${SERVER_ADDRESS}" "localhost" "8443" "9443" || {
-    
+          ssh -o "StrictHostKeyChecking=no" -o "UserKnownHostsFile=/dev/null" "root@${SERVER_ADDRESS}" cat /usr/local/etc/instance.cfg
+          test-ncp-instance -a -f "$SNAPSHOT_ID" -b "v1.54.3" --systemtest-args "--skip-update-test" --nc-test-args "--skip-release-check" "root@${SERVER_ADDRESS}" "localhost" "8443" "9443" || {
+          
             echo "Integration tests failed"
             echo "Here are the last lines of ncp-install.log:"
             echo "==========================================="
@@ -158,66 +354,40 @@ jobs:
             echo "==========================================="
             exit 1
           }
-
-  run-update-test:
-    needs:
-      - setup-update-test-instance
-    runs-on: ubuntu-latest
-
-    container:
-      image: thecalcaholic/ncp-test-automation:latest
-      env:
-        HCLOUD_TOKEN: "${{ secrets.TEST_AUTOMATION_HCLOUD_API_TOKEN }}"
-        UID: "${{ github.run_id }}-update"
-    env:
-      PREVIOUS_VERSION: ${{ needs.setup-update-test-instance.outputs.previous_version }}
-      VERSION: ${{ needs.setup-update-test-instance.outputs.version }}
-      SERVER_ADDRESS: "${{ needs.setup-update-test-instance.outputs.server_address }}"
-      SNAPSHOT_ID: "${{ needs.setup-update-test-instance.outputs.snapshot_id }}"
-      HOME: /root
-    defaults:
-      run:
-        shell: bash
-    steps:
-      - uses: actions/checkout@v3
-        with:
-          repository: 'theCalcaholic/ncp-test-automation'
-      - name: download ssh private key from artifact store
-        uses: actions/download-artifact@v3
-        with:
-          name: ${{ github.run_id }}-update-ssh-privkey
-          path: .ssh
       - name: perform update
+        working-directory: /__w/nextcloudpi/nextcloudpi/ncp-test-automation/bin
         run: |
           set -e
           
           echo "Setup ssh"
-          chmod 0600 ./.ssh/automation_ssh_key
+          chmod 0600 /__w/nextcloudpi/nextcloudpi/.ssh/automation_ssh_key
           eval "$(ssh-agent)"
-          ssh-add ./.ssh/automation_ssh_key
+          ssh-add /__w/nextcloudpi/nextcloudpi/.ssh/automation_ssh_key
           
-          . ./bin/library.sh
+          source ./library.sh
 
           echo "Updating from $PREVIOUS_VERSION to $VERSION"
           ssh-keygen -f "$HOME/.ssh/known_hosts" -R "${SERVER_ADDRESS}" 2> /dev/null || true
           ssh "${SSH_OPTIONS[@]}" "root@${SERVER_ADDRESS}" "ncp-update '$VERSION'"
-      - name: Run integration tests
+      - name: Run integration tests after update
+        working-directory: /__w/nextcloudpi/nextcloudpi/ncp-test-automation/bin
         run: |
           set -e
           
           echo "Setup ssh"
           eval "$(ssh-agent)"
-          ssh-add ./.ssh/automation_ssh_key
+          ssh-add /__w/nextcloudpi/nextcloudpi/.ssh/automation_ssh_key
           
-          cd bin
           source ./library.sh
           
           trap 'terminate-ssh-port-forwarding "${SERVER_ADDRESS}"' EXIT 1 2
           
           echo "Run integration tests"
           setup-ssh-port-forwarding "$SERVER_ADDRESS"
-          
-          test-ncp-instance -a -f "$SNAPSHOT_ID" -b "${VERSION}" "root@${SERVER_ADDRESS}" "localhost" "8443" "9443" || {
+          NC_TEST_ARGS=()
+          ssh "${SSH_OPTIONS[@]}" "root@${SERVER_ADDRESS}" cat /etc/os-release | grep VERSION_ID=12 || NC_TEST_ARGS+=("--skip-release-check")
+          set -x
+          test-ncp-instance -f "$SNAPSHOT_ID" -b "v1.54.3" --nc-test-args "$NC_TEST_ARGS" --systemtest-args "--skip-update-test" "root@${SERVER_ADDRESS}" "localhost" "8443" "9443" || {
           
             echo "Integration tests failed"
             echo "Here are the last lines of ncp-install.log:"
@@ -231,11 +401,103 @@ jobs:
             exit 1
           }
 
+      - name: NCP distupgrade
+        id: distupgrade
+        working-directory: /__w/nextcloudpi/nextcloudpi/ncp-test-automation/bin
+        run: |
+          set -e
+          
+          echo "Setup ssh"
+          eval "$(ssh-agent)"
+          ssh-add /__w/nextcloudpi/nextcloudpi/.ssh/automation_ssh_key
+          
+          source ./library.sh
+          
+          ssh "${SSH_OPTIONS[@]}" "root@${SERVER_ADDRESS}" cat /etc/os-release | grep 'VERSION_ID="11"' || {
+            echo "Can't upgrade from Debian $(ssh "${SSH_OPTIONS[@]}" "root@${SERVER_ADDRESS}" grep 'VERSION_ID=' /etc/os-release)"
+            echo "skipped=yes" | tee -a $GITHUB_OUTPUT
+            exit 1
+          }
+          ssh "${SSH_OPTIONS[@]}" "root@${SERVER_ADDRESS}" DEBIAN_FRONTEND=noninteractive ncp-dist-upgrade
+          echo "skipped=no" | tee -a $GITHUB_OUTPUT
+      - name: Update Nextcloud
+        working-directory: /__w/nextcloudpi/nextcloudpi/ncp-test-automation/bin
+        run: |
+          set -ex
+          apk add jq
+          
+          echo "Setup ssh"
+          eval "$(ssh-agent)"
+          ssh-add /__w/nextcloudpi/nextcloudpi/.ssh/automation_ssh_key
+          
+          source ./library.sh
+          
+          current_nc_version="$(ssh "${SSH_OPTIONS[@]}" "root@${SERVER_ADDRESS}" "ncc status" | grep "version:" | awk '{ print $3 }')"
+          latest_nc_version="$(cat ../../etc/ncp.cfg | jq -r '.nextcloud_version')"
+          
+          if [[ "$current_nc_version" =~ "$latest_nc_version".* ]]
+          then
+            echo "Nextcloud is up to date - skipping NC update test."
+          else
+            for i in {1..10};
+            do
+              echo "running nc update ($i/10)..."
+              ssh "${SSH_OPTIONS[@]}" "root@${SERVER_ADDRESS}" "bash -c 'DBG=x ncp-update-nc ${latest_nc_version?}'"
+          
+              ssh "${SSH_OPTIONS[@]}" "root@${SERVER_ADDRESS}" /usr/local/bin/ncc status
+          
+              current_nc_version_new="$(ssh "${SSH_OPTIONS[@]}" "root@${SERVER_ADDRESS}" "ncc status" | grep "version:" | awk '{ print $3 }')"
+              if [[ "$current_nc_version_new" =~ "$latest_nc_version".* ]]
+              then
+                break
+              fi
+              if [[ "$current_nc_version" == "$current_nc_version_new" ]]
+              then
+                echo "failed to update to $latest_nc_version"
+                exit 1
+              fi
+              current_nc_version="$current_nc_version_new"
+            done
+          fi
+
+      - name: Run integration tests after dist-upgrade
+        id: final_test
+        working-directory: /ncp-test-automation/bin
+        run: |
+          set -e
+          
+          echo "Setup ssh"
+          eval "$(ssh-agent)"
+          ssh-add /__w/nextcloudpi/nextcloudpi/.ssh/automation_ssh_key
+          
+          source ./library.sh
+          
+          trap '[ $? -eq 0 ] || echo "test_result=failure" >> "$GITHUB_OUTPUT"; terminate-ssh-port-forwarding "${SERVER_ADDRESS}"' EXIT 1 2
+          
+          echo "Run integration tests"
+          setup-ssh-port-forwarding "$SERVER_ADDRESS"
+          
+          test-ncp-instance -f "$SNAPSHOT_ID" -b "${VERSION}" --systemtest-args "--skip-update-test" "root@${SERVER_ADDRESS}" "localhost" "8443" "9443" || {
+          
+            echo "Integration tests failed"
+            echo "Here are the last lines of ncp-install.log:"
+            echo "==========================================="
+            ssh "${SSH_OPTIONS[@]}" "root@${SERVER_ADDRESS}" tail /var/log/ncp-install.log;
+            echo "==========================================="
+            echo "ncp.log:"
+            echo "==========================================="
+            ssh "${SSH_OPTIONS[@]}" "root@${SERVER_ADDRESS}" tail /var/log/ncp.log;
+            echo "==========================================="
+            echo "nextcloud.log:"
+            ssh "${SSH_OPTIONS[@]}" "root@${SERVER_ADDRESS}" tail /opt/ncdata/data/nextcloud.log;
+            exit 1
+          }
+          echo "test_result=success" >> "$GITHUB_OUTPUT"
+
   install-postactivation-snapshot:
     if: ${{ always() }}
     needs:
-      - setup-installation-test-instance
-      - run-installation-test
+      - installation-test
     runs-on: ubuntu-latest
     container:
       image: thecalcaholic/ncp-test-automation:latest
@@ -244,28 +506,28 @@ jobs:
 
     env:
       TEST_TYPE: install
-      SERVER_ADDRESS: ${{ needs.setup-installation-test-instance.outputs.server_address }}
-      TEST_RESULT: ${{ needs.setup-installation-test-instance.result }}
-      TEST_SERVER_ID: ${{ needs.setup-installation-test-instance.outputs.test_server_id }}
-      VERSION: ${{ needs.setup-installation-test-instance.outputs.version }}
-
+      SERVER_ADDRESS: ${{ needs.installation-test.outputs.server_address }}
+      TEST_RESULT: ${{ needs.installation-test.test_result }}
+      TEST_SERVER_ID: ${{ needs.installation-test.outputs.test_server_id }}
+      VERSION: ${{ needs.installation-test.outputs.version }}
+      SSH_ARTIFACT_NAME: "${{ needs.installation-test.outputs.ssh_artifact_name }}"
       UID: ${{ github.run_id }}-install
     steps:
       - name: download ssh private key from artifact store
-        uses: actions/download-artifact@v3
+        uses: actions/download-artifact@v4
         if: ${{ contains('success|failure', env.TEST_RESULT) }}
         with:
-          name: ${{ github.run_id }}-${{ env.TEST_TYPE }}-ssh-privkey
+          name: ${{ env.SSH_ARTIFACT_NAME }}
           path: /github/workspace/.ssh
       - name: Shutdown server
         if: ${{ contains('success|failure', env.TEST_RESULT) }}
         run: |
-          chmod 0600 /github/workspace/.ssh/automation_ssh_key
-          export SSH_PUBLIC_KEY="$(cat /github/workspace/.ssh/automation_ssh_key.pub)"
+          chmod 0600  /github/workspace/.ssh/automation_ssh_key
+          export SSH_PUBLIC_KEY="$(cat  /github/workspace/.ssh/automation_ssh_key.pub)"
           bash /ncp-test-automation/bin/entrypoint.sh
           eval "$(ssh-agent)"
-          ssh-add /github/workspace/.ssh/automation_ssh_key
-  
+          ssh-add  /github/workspace/.ssh/automation_ssh_key
+          
           ssh -o "StrictHostKeyChecking=no" -o "UserKnownHostsFile=/dev/null" "root@${SERVER_ADDRESS?}" <<EOF
           systemctl stop mariadb
           systemctl poweroff
@@ -273,11 +535,10 @@ jobs:
       - name: Create Snapshot
         if: ${{ contains('success|failure', env.TEST_RESULT) }}
         shell: bash
+        working-directory: /ncp-test-automation/bin
         run: |
           set -x
-          echo "${{ needs.setup-installation-test-instance.outputs.test_server_id }}"
           echo "${TEST_SERVER_ID?}"
-          cd /ncp-test-automation/bin
           
           . ./library.sh
           
@@ -286,11 +547,64 @@ jobs:
           snapshot_id="$(tf-output "$TF_SNAPSHOT" -state="${TF_SNAPSHOT}/${VERSION//\//.}.postactivation.tfstate" snapshot_id)"
           hcloud image add-label -o "$snapshot_id" "test-result=${TEST_RESULT?}"
 
-  update-postactivation-snapshot:
+#  update-postactivation-snapshot:
+#    if: ${{ always() }}
+#    needs:
+#      - update-test
+#    runs-on: ubuntu-latest
+#    container:
+#      image: thecalcaholic/ncp-test-automation:latest
+#      env:
+#        HCLOUD_TOKEN: "${{ secrets.TEST_AUTOMATION_HCLOUD_API_TOKEN }}"
+#
+#    env:
+#      TEST_TYPE: update
+#      SERVER_ADDRESS: ${{ needs.update-test.outputs.server_address }}
+#      TEST_RESULT: ${{ needs.update-test.test_result }}
+#      TEST_SERVER_ID: ${{ needs.update-test.outputs.test_server_id }}
+#      VERSION: ${{ needs.update-test.outputs.version }}
+#      UID: ${{ github.run_id }}-update
+#      SSH_ARTIFACT_NAME: "${{ needs.update-test.outputs.ssh_artifact_name }}"
+#    steps:
+#      - name: download ssh private key from artifact store
+#        uses: actions/download-artifact@v3
+#        if: ${{ contains('success|failure', env.TEST_RESULT) }}
+#        with:
+#          name: ${{ env.SSH_ARTIFACT_NAME }}
+#          path: /github/workspace/.ssh
+#      - name: Shutdown server
+#        if: ${{ contains('success|failure', env.TEST_RESULT) }}
+#        run: |
+#          chmod 0600  /github/workspace/.ssh/automation_ssh_key
+#          export SSH_PUBLIC_KEY="$(cat  /github/workspace/.ssh/automation_ssh_key.pub)"
+#          bash /ncp-test-automation/bin/entrypoint.sh
+#          eval "$(ssh-agent)"
+#          ssh-add  /github/workspace/.ssh/automation_ssh_key
+#
+#          ssh -o "StrictHostKeyChecking=no" -o "UserKnownHostsFile=/dev/null" "root@${SERVER_ADDRESS?}" <<EOF
+#          systemctl stop mariadb
+#          systemctl poweroff
+#          EOF
+#      - name: Create Snapshot
+#        if: ${{ contains('success|failure', env.TEST_RESULT) }}
+#        shell: bash
+#        working-directory: /ncp-test-automation/bin
+#        run: |
+#          set -x
+#          echo "${{ needs.update-test.outputs.test_server_id }}"
+#          echo "${TEST_SERVER_ID?}"
+#
+#          . ./library.sh
+#
+#          tf-init "$TF_SNAPSHOT"
+#          tf-apply "$TF_SNAPSHOT" "$TF_VAR_FILE" -var="branch=${VERSION?}" -var="snapshot_provider_id=${TEST_SERVER_ID?}" -var="snapshot_type=ncp-postactivation" -state="${TF_SNAPSHOT}/${VERSION//\//.}.postactivation.tfstate"
+#          snapshot_id="$(tf-output "$TF_SNAPSHOT" -state="${TF_SNAPSHOT}/${VERSION//\//.}.postactivation.tfstate" snapshot_id)"
+#          hcloud image add-label -o "$snapshot_id" "test-result=${TEST_RESULT?}"
+
+  dist-upgrade-postactivation-snapshot:
     if: ${{ always() }}
     needs:
-      - setup-update-test-instance
-      - run-update-test
+      - dist-upgrade-test
     runs-on: ubuntu-latest
     container:
       image: thecalcaholic/ncp-test-automation:latest
@@ -298,27 +612,28 @@ jobs:
         HCLOUD_TOKEN: "${{ secrets.TEST_AUTOMATION_HCLOUD_API_TOKEN }}"
 
     env:
-      TEST_TYPE: update
-      SERVER_ADDRESS: ${{ needs.setup-update-test-instance.outputs.server_address }}
-      TEST_RESULT: ${{ needs.setup-update-test-instance.result }}
-      TEST_SERVER_ID: ${{ needs.setup-update-test-instance.outputs.test_server_id }}
-      VERSION: ${{ needs.setup-update-test-instance.outputs.version }}
-      UID: ${{ github.run_id }}-update
+      TEST_TYPE: distupgrade
+      SERVER_ADDRESS: ${{ needs.dist-upgrade-test.outputs.server_address }}
+      TEST_RESULT: ${{ needs.dist-upgrade-test.outputs.test_result }}
+      TEST_SERVER_ID: ${{ needs.dist-upgrade-test.outputs.test_server_id }}
+      VERSION: ${{ needs.dist-upgrade-test.outputs.version }}
+      UID: ${{ github.run_id }}-distupgrade
+      SSH_ARTIFACT_NAME: "${{ needs.dist-upgrade-test.outputs.ssh_artifact_name }}"
     steps:
       - name: download ssh private key from artifact store
-        uses: actions/download-artifact@v3
+        uses: actions/download-artifact@v4
         if: ${{ contains('success|failure', env.TEST_RESULT) }}
         with:
-          name: ${{ github.run_id }}-${{ env.TEST_TYPE }}-ssh-privkey
+          name: ${{ env.SSH_ARTIFACT_NAME }}
           path: /github/workspace/.ssh
       - name: Shutdown server
         if: ${{ contains('success|failure', env.TEST_RESULT) }}
         run: |
-          chmod 0600 /github/workspace/.ssh/automation_ssh_key
-          export SSH_PUBLIC_KEY="$(cat /github/workspace/.ssh/automation_ssh_key.pub)"
+          chmod 0600  /github/workspace/.ssh/automation_ssh_key
+          export SSH_PUBLIC_KEY="$(cat  /github/workspace/.ssh/automation_ssh_key.pub)"
           bash /ncp-test-automation/bin/entrypoint.sh
           eval "$(ssh-agent)"
-          ssh-add /github/workspace/.ssh/automation_ssh_key
+          ssh-add  /github/workspace/.ssh/automation_ssh_key
           
           ssh -o "StrictHostKeyChecking=no" -o "UserKnownHostsFile=/dev/null" "root@${SERVER_ADDRESS?}" <<EOF
           systemctl stop mariadb
@@ -327,11 +642,11 @@ jobs:
       - name: Create Snapshot
         if: ${{ contains('success|failure', env.TEST_RESULT) }}
         shell: bash
+        working-directory: /ncp-test-automation/bin
         run: |
           set -x
-          echo "${{ needs.setup-update-test-instance.outputs.test_server_id }}"
+          echo "${{ needs.dist-upgrade-test.outputs.test_server_id }}"
           echo "${TEST_SERVER_ID?}"
-          cd /ncp-test-automation/bin
           
           . ./library.sh
           
@@ -339,12 +654,13 @@ jobs:
           tf-apply "$TF_SNAPSHOT" "$TF_VAR_FILE" -var="branch=${VERSION?}" -var="snapshot_provider_id=${TEST_SERVER_ID?}" -var="snapshot_type=ncp-postactivation" -state="${TF_SNAPSHOT}/${VERSION//\//.}.postactivation.tfstate"
           snapshot_id="$(tf-output "$TF_SNAPSHOT" -state="${TF_SNAPSHOT}/${VERSION//\//.}.postactivation.tfstate" snapshot_id)"
           hcloud image add-label -o "$snapshot_id" "test-result=${TEST_RESULT?}"
-
+  
   cleanup:
     if: ${{ always() }}
     needs:
       - install-postactivation-snapshot
-      - update-postactivation-snapshot
+      #- update-postactivation-snapshot
+      - dist-upgrade-postactivation-snapshot
     runs-on: ubuntu-latest
     container:
       image: thecalcaholic/ncp-test-automation:latest
@@ -352,7 +668,7 @@ jobs:
         HCLOUD_TOKEN: "${{ secrets.TEST_AUTOMATION_HCLOUD_API_TOKEN }}"
     strategy:
       matrix:
-        uid: ["${{ github.run_id }}-install", "${{ github.run_id }}-update"]
+        uid: ["${{ github.run_id }}-install", "${{ github.run_id }}-update", "${{ github.run_id }}-distupgrade"]
       fail-fast: false
     env:
       HOME: '/root'
@@ -378,7 +694,8 @@ jobs:
     if: ${{ always() }}
     needs:
       - install-postactivation-snapshot
-      - update-postactivation-snapshot
+      #- update-postactivation-snapshot
+      - dist-upgrade-postactivation-snapshot
     runs-on: ubuntu-latest
     container:
       image: thecalcaholic/ncp-test-automation:latest
@@ -389,7 +706,7 @@ jobs:
     steps:
       - name: Delete old snapshots
         run: |
-          for snapshot in $(hcloud image list -t snapshot -o noheader -o columns=id | head -n -16)
+          for snapshot in $(hcloud image list -t snapshot -o noheader -o columns=id | head -n -12)
           do
             echo "Deleting snapshot '$snapshot'..."
             hcloud image delete "$snapshot"

.gitignore

@@ -11,3 +11,4 @@
 /ncp-web/ncp-web.cfg
 /docker-armhf/qemu-arm-static
 .vagrant/
+/.idea/

FUNDING.yml

@@ -0,0 +1,14 @@
+# These are supported funding model platforms
+
+github: # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2]
+patreon: # Replace with a single Patreon username
+open_collective: # Replace with a single Open Collective username
+ko_fi: thecalcaholic
+tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
+community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
+liberapay: # Replace with a single Liberapay username
+issuehunt: # Replace with a single IssueHunt username
+lfx_crowdfunding: # Replace with a single LFX Crowdfunding project-name e.g., cloud-foundry
+polar: # Replace with a single Polar username
+buy_me_a_coffee: # Replace with a single Buy Me a Coffee username
+custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2']

README.md

@@ -4,7 +4,7 @@ _(The translated README pages are not updated at this time)_
 
 # NextcloudPi
 
-[![Telegram icon][telegram-badge]][chat-telegram] [![Matrix icon][matrix-badge]][chat-matrix] [![Docker icon][docker-badge]][ncp-docker-hub] [![Nextcloud icon][nc-badge]][nc-github]
+[![Telegram icon][telegram-badge]][chat-telegram] [![Matrix icon][matrix-badge]][chat-matrix]  [![Nextcloud icon][nc-badge]][nc-github]
 
 [![Forum icon][forum-badge]][nc-forum-support]
 
@@ -19,7 +19,7 @@ This is the build code for the [NextcloudPi][ncp-website] open-source community
 
 NextcloudPi is a ready to use image for Virtual Machines, Raspberry Pi, Odroid HC1, Rock64 and other boards. ([⇒ Downloads][ncp-releases])
 
-This code also generates the NextcloudPi [Docker image][ncp-docker-hub], LXD container & VM, there is an install script for the latest stable Debian based system as well.
+This code also generates the NextcloudPi LXD and LXC containers and there is an install script for the latest supported Debian based system as well.
 
 Find the documentation at [docs.nextcloudpi.com][ncp-docs-website], the documentation is all written by volunteers.
 
@@ -27,23 +27,21 @@ Please reach out in the [Matrix][chat-matrix-wiki] or [Telegram][chat-telegram-w
 
 ---
 
+### Test Status
+
 `master`
 
-[![VM Tests][vm-tests-badge]][vm-tests]
-
-[![Docker Tests][docker-tests-badge]][docker-tests]
+[![Release](https://github.com/nextcloud/nextcloudpi/actions/workflows/release.yml/badge.svg)](https://github.com/nextcloud/nextcloudpi/actions/workflows/release.yml)
 
 `devel`
 
-[![VM Tests][gh-vm-tests-badge-devel]][vm-tests]
-
-[![Docker Tests][gh-docker-tests-badge-devel]][docker-tests]
+[![Release](https://github.com/nextcloud/nextcloudpi/actions/workflows/release.yml/badge.svg?branch=devel)](https://github.com/nextcloud/nextcloudpi/actions/workflows/release.yml)
 
 ---
 
 ## Features
 
- * Raspberry Pi OS/Debian 11 _(Bullseye)_
+ * Raspberry Pi OS/Debian 12 _(Bookworm)_
  * Nextcloud
  * Apache, with HTTP2 enabled
  * PHP 8.1
@@ -108,35 +106,9 @@ sudo ncp-config
 
 ![NCP-config][ncp-config-image]
 
+## Docker has been discontinued
 
-## Run in docker
-
-```
-docker run --detach \
-           --publish 4443:4443 \
-           --publish 443:443 \
-           --publish 80:80 \
-           --volume ncdata:/data \
-           --name nextcloudpi \
-           ownyourbits/nextcloudpi $DOMAIN
-```
-
-`$DOMAIN` can also be the IP-address of the host device if you're accessing it via IP-address in your local home network.
-
-Can also be run with the `--init` flag for zombie process reaping
-
-```
-docker run --detach \
-           --init \
-           --publish 4443:4443 \
-           --publish 443:443 \
-           --publish 80:80 \
-           --volume ncdata:/data \
-           --name nextcloudpi \
-           ownyourbits/nextcloudpi $DOMAIN
-```
-
-It takes a moment to start completely, you can check this with `docker logs nextcloudpi` until it says `Init done`.
+Docker has been discontinued for the time being, please read the announcement here: https://help.nextcloud.com/t/nextcloudpi-planning-to-discontinue-its-docker-version-with-nc-25/158895
 
 ## Run in LXD
 
@@ -153,19 +125,35 @@ lxc start ncp
 
 ## Run in Proxmox
 
-Use the [install script][ncp-proxmox-install-script-v5] from [tteck][tteck-profile] to install the LXC container on your Proxmox instance
+There are two ways to run NCP on Proxmox:
 
-He has multiple helper scripts available for Proxmox on his [website][website-helper-scripts], do go have a look if you're using Proxmox. :+1:
+### 1. Using the official release image
 
-Installation: `bash -c "$(wget -qLO - https://github.com/tteck/Proxmox/raw/main/ct/nextcloudpi-v5.sh)"`
+In your Proxmox web interface, head to any storage pool that has CT Templates enabled, go to CT Templates, 
+click "Download from URL" and enter the link to the latest LXC image from the [Nextcloudpi releases page](https://github.com/nextcloud/nextcloudpi/releases)
+and press "Download".
+
+<img height="300" alt="Download from URL dialog" src="https://nextcloudpi.com/images/ncp-proxmox-download-screenshot.png">
+
+Optionally, you can copy the corresponding md5 sum from the releases page and paste it in the "Checksum" field to have
+Proxmox check that the downloaded image was not corrupted
+
+### 2. Installer by community-scripts (formerly tteck)
+
+Use the [install script][ncp-proxmox-install-script-v5] from [community-scripts][community-scripts-profile] to install the LXC container on your Proxmox instance
+
+[community-scripts][community-scripts-profile] has multiple helper scripts available for Proxmox on their [website][website-helper-scripts], go have a look if you're using Proxmox. :+1:
+
+
+Installation: `bash -c "$(curl -fsSL https://raw.githubusercontent.com/community-scripts/ProxmoxVE/main/ct/nextcloudpi.sh)"`
 
 Default Settings: `2GB RAM - 8GB Storage - 2vCPU`
 
-_(Check his [website][website-helper-scripts] if this has changed and we haven't had the time to update it here yet, it's located under: Media - Photo > NextcloudPi LXC)_
+_(Check their [website][website-helper-scripts] if this has changed and we haven't had the time to update it here yet, it's located under: Operating-Systems > NextcloudPi LXC)_
 
-Thenk you [tteck][tteck-profile] :heart: for making the helper script & letting us use this for Proxmox installations :pray:
+Thank you to the [community-scripts][community-scripts-profile] organisation :heart: for making the helper script & letting us use this for Proxmox installations :pray:
 
-You can find his GitHub repository with his helper scripts [here][gh-helper-scripts-repo].
+You can find the GitHub repository with the helper scripts here: [here][gh-helper-scripts-repo].
 
 ## How to build
 
@@ -191,8 +179,6 @@ Packages
 - `whiptail`
 - `qemu`
 - `qemu-user-static`
-- `docker` _(If you're building a Docker image)_
-- `lxd` _(If you're building an LXD/LXC container image)_
 
 ### Raspberry Pi IMG
 
@@ -208,16 +194,6 @@ cd nextcloudpi
 ./build-SD-armbian.sh odroidxu4   # supported board code name
 ```
 
-In order to build & push the Docker image to your repository, you'll also need to change the username, repo and tags in the script to match your credentials at Docker Hub.
-
-```
-git clone https://github.com/nextcloud/nextcloudpi.git
-cd nextcloudpi
-build/build-docker.sh x86
-build/build-docker.sh armhf
-build/build-docker.sh arm64
-```
-
 ### LXD
 
 ```
@@ -248,7 +224,7 @@ curl -sSL https://raw.githubusercontent.com/nextcloud/nextcloudpi/master/install
 
 [Downloads][ncp-releases]
 
-[Docker Hub][ncp-docker-hub]
+<!-- [Docker Hub][ncp-docker-hub] -->
 
 [Nextcloud Forum][nc-forum]
 
@@ -310,8 +286,6 @@ You can find us on the [Forum][nc-forum], [Telegram][chat-telegram] or [Matrix][
 
 [docker-tests-badge]: https://github.com/nextcloud/nextcloudpi/actions/workflows/build-docker.yml/badge.svg
 
-[docker-badge]: https://img.shields.io/badge/Docker-2CA5E0?style=for-the-badge&logo=docker&logoColor=white
-
 [telegram-badge]: https://img.shields.io/badge/Telegram-2CA5E0?style=for-the-badge&logo=telegram&logoColor=white
 
 [matrix-badge]: https://img.shields.io/badge/matrix-000000?style=for-the-badge&logo=Matrix&logoColor=white
@@ -320,15 +294,15 @@ You can find us on the [Forum][nc-forum], [Telegram][chat-telegram] or [Matrix][
 
 [nc-badge]: https://img.shields.io/badge/Nextcloud-0082C9?style=for-the-badge&logo=Nextcloud&logoColor=white
 
-<!-- TTECK -->
+<!-- COMMUNITY-SCRIPTS -->
 
-[tteck-profile]: https://github.com/tteck
+[community-scripts-profile]: https://github.com/community-scripts/
 
-[gh-helper-scripts-repo]: https://github.com/tteck/Proxmox
+[gh-helper-scripts-repo]: https://github.com/community-scripts/ProxmoxVE
 
-[website-helper-scripts]: https://tteck.github.io/Proxmox/
+[website-helper-scripts]: https://community-scripts.github.io/ProxmoxVE/
 
-[ncp-proxmox-install-script-v5]: https://github.com/tteck/Proxmox/blob/main/install/nextcloudpi-v5-install.sh
+[ncp-proxmox-install-script-v5]: https://github.com/community-scripts/ProxmoxVE/blob/main/install/nextcloudpi-install.sh
 
 <!-- IMAGES -->
 

bin/ncp-check-nc-version

@@ -6,18 +6,23 @@ set -e
 
 source /usr/local/etc/library.sh # sets NCLATESTVER
 
-CURRENT="$(ncc status | grep "version:" | awk '{ print $3 }')"
-LATEST="$(wget -qO- https://raw.githubusercontent.com/nextcloud/nextcloudpi/master/etc/ncp.cfg | jq -r .nextcloud_version)"
+CURRENT="$(nc_version)"
+NEXT_VERSION="$(determine_nc_update_version "${CURRENT}" "${NCLATESTVER?}")"
+if [[ -z "$NEXT_VERSION" ]] || [[ "$NEXT_VERSION" == "${CURRENT}" ]]
+then
+  exit 0
+fi
+
 NOTIFIED=/var/run/.nc-version-notified
 
-test -e "${NOTIFIED}" && [[ "${LATEST}" == "$( cat "${NOTIFIED}" )" ]] && {
-  echo "Found update from ${CURRENT} to ${LATEST}. Already notified"
+test -e "${NOTIFIED}" && [[ "${NEXT_VERSION}" == "$( cat "${NOTIFIED}" )" ]] && {
+  echo "Found update from ${CURRENT} to ${NEXT_VERSION}. Already notified"
   exit 0
 }
 
-if is_more_recent_than "${LATEST}" "${CURRENT}"; then
+if is_more_recent_than "${NEXT_VERSION}" "${CURRENT}"; then
   notify_admin \
     "Nextcloud update" \
-    "Update from ${CURRENT} to ${LATEST} is available. Update from https://$(get_ip):4443"
-  echo "${LATEST}" > "${NOTIFIED}"
+    "Update from ${CURRENT} to ${NEXT_VERSION} is available. Update from https://$(get_ip):4443"
+  echo "${NEXT_VERSION}" > "${NOTIFIED}"
 fi

bin/ncp-check-updates

@@ -6,8 +6,8 @@ VERFILE=/usr/local/etc/ncp-version
 LATEST=/var/run/.ncp-latest-version
 
 if ncp-test-updates; then
-  echo -e "\nNextCloudPi \e[1m$( cat $VERFILE)\e[0m is outdated"
+  echo -e "\nNextcloudPi \e[1m$( cat $VERFILE)\e[0m is outdated"
   echo -e "update to \e[1m$( cat $LATEST )\e[0m through 'ncp-config' or type 'sudo ncp-update'" 
 else
-  echo -e "\nNextCloudPi \e[1m$( cat $VERFILE)\e[0m is up to date"
+  echo -e "\nNextcloudPi \e[1m$( cat $VERFILE)\e[0m is up to date"
 fi

bin/ncp-check-version

@@ -2,19 +2,33 @@
 
 # update latest available version in /var/run/.ncp-latest-version
 
-TMPDIR="$( mktemp -d /tmp/ncp-check.XXXXXX || ( echo "Failed to create temp dir. Exiting" >&2; exit 1 ) )"
-trap "rm -rf \"${TMPDIR}\"; exit 0" 0 1 2 3 15
+TEMPDIR="$( mktemp -d /tmp/ncp-check.XXXXXX || ( echo "Failed to create temp dir. Exiting" >&2; exit 1 ) )"
+trap "rm -rf \"${TEMPDIR}\"; exit 0" 0 1 2 3 15
 
 BRANCH="master"
-is_docker && BRANCH="docker-stable"
+{ [[ -f /.dockerenv ]] || [[ -f /.docker-image ]] || [[ "$DOCKERBUILD" == 1 ]]; } && BRANCH="docker-stable"
 
-git clone -b "$BRANCH" --depth 20 -q --bare https://github.com/nextcloud/nextcloudpi.git "$TMPDIR" || {
+git clone -b "$BRANCH" --depth 20 -q --bare https://github.com/nextcloud/nextcloudpi.git "$TEMPDIR" || {
   echo "The git clone command failed: No connectivity to https://github.com ?" >&2
   exit 1
 }
 
-cd "$TMPDIR" || exit 1
+cd "$TEMPDIR" || exit 1
 VER=$( git describe --always --tags | grep -oP "v\d+\.\d+\.\d+" )
+
+canary="$(. /usr/local/etc/library.sh; find_app_param ncp-community.sh CANARY)"
+
+[[ "$canary" != "yes" ]] && [[ -f "/usr/local/etc/instance.cfg" ]] && {
+  cohorte_id="$(jq .cohorteId /usr/local/etc/instance.cfg)"
+  [[ -f "./staged_rollouts/${VER}.txt" ]] && {
+    grep "^${cohorte_id}$" "./staged_rollouts/${VER}.txt" || {
+      echo "Skipping version $VER - cohorte ${cohorte_id} not yet active"
+      cd / || exit 1
+      exit 0
+    }
+  }
+}
+
 grep -qP "v\d+\.\d+\.\d+" <<< "$VER" && {       # check format
   echo "$VER" > /var/run/.ncp-latest-version
 

bin/ncp-config

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# NextCloudPi software configuration
+# NextcloudPi software configuration
 #
 # Copyleft 2017 by Ignacio Nunez Hernanz <nacho _a_t_ ownyourbits _d_o_t_ com>
 # GPL licensed (see end of file) * Use at your own risk!
@@ -23,7 +23,7 @@ source /usr/local/etc/library.sh
 
     latest_ver="$(cat /var/run/.ncp-latest-version 2>/dev/null)"
     whiptail --backtitle "$backtitle $ncpversion" \
-             --title "NextCloudPi update available" \
+             --title "NextcloudPi update available" \
              --clear --yesno "Update to $latest_ver?\n\n$changelog" \
              15 70
 
@@ -59,7 +59,7 @@ function generate_list()
 function config_menu()
 {
   local dir="$1"
-  local backtitle="NextCloudPi configuration ver. "
+  local backtitle="NextcloudPi configuration ver. "
   local ncpversion="$(cat /usr/local/etc/ncp-version  )"
   local cfgdir=/usr/local/etc/ncp-config.d
   local dialog_ok=0
@@ -73,7 +73,7 @@ function config_menu()
     # launch the selection menu
     [[ "$dir" == "$BINDIR" ]] && local cancel_btn="Finish" || local cancel_btn="Back"
     ncp_app=$( whiptail --backtitle "$backtitle $ncpversion" \
-                       --title "NextCloudPi Configuration Tool (ncp-config)" \
+                       --title "NextcloudPi Configuration Tool (ncp-config)" \
                        --cancel-button $cancel_btn --ok-button Select \
                        --menu "Select ncp-app to configure or activate:" 20 105 10 \
                        "${list[@]}" \

bin/ncp-diag

@@ -151,7 +151,7 @@ echo "Gateway|$GW"
 echo "Interface|$IFACE"
 
 # Certificates
-CERTS="$( grep "SSLCertificateFile */etc/letsencrypt/live/" /etc/apache2/sites-available/nextcloud.conf \
+CERTS="$( grep "SSLCertificateFile */etc/letsencrypt/live/" /etc/apache2/sites-available/001-nextcloud.conf \
         | sed 's|.*SSLCertificateFile */etc/letsencrypt/live/||;s|/fullchain.pem||' )"
 [[ "$CERTS" == "" ]] && CERTS=none
 echo "Certificates|$CERTS"

bin/ncp-dist-upgrade

@@ -1,137 +1,35 @@
-#!/bin/bash
+#!/usr/bin/env bash
 
-set -eu -o pipefail
-
-[[ -f /.dockerenv ]] && { echo "Not supported in Docker. Upgrade the container instead"; exit 0; }
-
-new_cfg=/usr/local/etc/ncp-recommended.cfg
-[[ -f "${new_cfg}" ]] || { echo "Already on the lastest recommended distribution. Abort." >&2; exit 1; }
-
-APTINSTALL="apt-get install -y --no-install-recommends"
-export DEBIAN_FRONTEND=noninteractive
-
-echo "
->>> ATTENTION <<<
-This is a dangerous process that is only guaranteed to work properly if you
-have not made manual changes in the system. Backup the SD card first and
-proceed at your own risk.
-
-Note that this is not a requirement for NCP to continue working properly.
-The current distribution will keep receiving updates for some time.
-
-Do you want to continue? [y/N]"
-
-read key
-[[ "$key" == y ]] || exit 0
-
-source /usr/local/etc/library.sh # sets NCPCFG RELEASE PHPVER
-old_cfg="${NCPCFG}"
-
-trap "echo 'Something went wrong. Fix it and try again'" EXIT
-
-save_maintenance_mode
-
-# Fix grub-pc issue in VM
-if apt show grub-pc-bin &>/dev/null; then
-  $APTINSTALL grub
-fi
-
-apt-get update
-apt-get upgrade -y
-
-# remove old PHP version
-set +e
-apt-get purge -y php${PHPVER} php${PHPVER}-curl php${PHPVER}-gd php${PHPVER}-fpm php${PHPVER}-cli php${PHPVER}-opcache \
-	php${PHPVER}-mbstring php${PHPVER}-xml php${PHPVER}-zip php${PHPVER}-fileinfo php${PHPVER}-ldap \
-	php${PHPVER}-intl php${PHPVER}-bz2 php${PHPVER}-json
-apt-get purge -y php${PHPVER}-mysql
-apt-get purge -y php${PHPVER}-redis
-apt-get purge -y php${PHPVER}-exif
-apt-get purge -y php${PHPVER}-bcmath
-apt-get purge -y php${PHPVER}-gmp
-apt-get purge -y php${PHPVER}-imagick
 set -e
 
-# update sources
-sed -i 's/buster/bullseye/g' /etc/apt/sources.list
-sed -i 's/buster/bullseye/g' /etc/apt/sources.list.d/* || true
-sed -i 's/bullseye\/updates/bullseye-security/g' /etc/apt/sources.list
-rm -f /etc/apt/sources.list.d/php.list
+[[ "${EUID}" -eq 0 ]] || {
+  echo "ERROR: Must be run as root (try sudo ncp-dist-upgrade)"
+  exit 1
+}
 
-# fix DHCP systemd service command https://forums.raspberrypi.com/viewtopic.php?t=320383 in raspbian
-if [[ -f /usr/bin/raspi-config ]]; then
-  sed -i 's|ExecStart=/usr/lib/dhcpcd5/dhcpcd -q -w|ExecStart=/usr/sbin/dhcpcd -q -w|g' /etc/systemd/system/dhcpcd.service.d/wait.conf
+. /etc/os-release
+
+if [[ "$VERSION_ID" -eq 10 ]]
+then
+  UPGRADE_CMD=(bash /usr/local/bin/ncp-dist-upgrade.d/debian-10.sh)
+elif [[ "$VERSION_ID" -eq 11 ]]
+then
+  UPGRADE_CMD=(bash /usr/local/bin/ncp-dist-upgrade.d/debian-11.sh)
+else
+  echo "No dist-upgrade available for OS version: Debian ${VERSION}."
+  exit 0
 fi
 
-# install latest distro
-apt-get update
-apt-get dist-upgrade -y
+if { [ "$TERM" = "screen" ] && [ -n "$TMUX" ]; } || [[ "${DEBIAN_FRONTEND:-}" == "noninteractive" ]] || ! [[ -t 0 ]]
+then
+  "${UPGRADE_CMD[@]}" |& tee /var/log/ncp-dist-upgrade.log
+else
+  tmux list-sessions | grep ncp-distupgrade && {
+    echo "Existing distupgrade process detected. Connecting..."
+    sleep 5
+    tmux attach -t ncp-distupgrade
+    exit 0
+  }
+  tmux new-session -s ncp-distupgrade bash -c "${UPGRADE_CMD[*]} |& tee /var/log/ncp-dist-upgrade.log; bash"
+fi
 
-# install latest PHP version
-release_new=$(jq -r '.release'     < "${new_cfg}")
-# the default repo in bullseye is bullseye-security - use bullseye if it is not available
-grep -Eh '^deb ' /etc/apt/sources.list | grep 'bullseye-security' > /dev/null && release_new="${release_new}-security"
-php_ver_new=$(jq -r '.php_version' < "${new_cfg}")
-# PHP 8.1 is only supported via the
-[[ "$php_ver_new" != 8.1 ]] || php_ver_new=7.4
-
-$APTINSTALL -t ${release_new} php${php_ver_new} php${php_ver_new}-curl php${php_ver_new}-gd php${php_ver_new}-fpm php${php_ver_new}-cli php${php_ver_new}-opcache \
-	php${php_ver_new}-mbstring php${php_ver_new}-xml php${php_ver_new}-zip php${php_ver_new}-fileinfo php${php_ver_new}-ldap \
-	php${php_ver_new}-intl php${php_ver_new}-bz2 php${php_ver_new}-json
-
-$APTINSTALL php${php_ver_new}-mysql
-$APTINSTALL -t ${release_new} php${php_ver_new}-redis
-
-$APTINSTALL -t ${release_new} smbclient exfat-fuse exfat-utils
-sleep 2 # avoid systemd thinking that PHP is in a crash/restart loop
-$APTINSTALL -t ${release_new} php${php_ver_new}-exif
-sleep 2 # avoid systemd thinking that PHP is in a crash/restart loop
-$APTINSTALL -t ${release_new} php${php_ver_new}-bcmath
-sleep 2 # avoid systemd thinking that PHP is in a crash/restart loop
-$APTINSTALL -t ${release_new} php${php_ver_new}-gmp
-#$APTINSTALL -t ${release_new} imagemagick php${php_ver_new}-imagick ghostscript
-
-# Reinstall prometheus-node-exporter, specifically WITH install-recommends to include collectors on bullseye and later
-{ dpkg -l | grep '^ii.*prometheus-node-exporter' >/dev/null && apt-get install -y prometheus-node-exporter-collectors; } || true
-
-apt-get autoremove -y
-apt-get clean
-
-cat > /etc/php/${php_ver_new}/fpm/conf.d/90-ncp.ini <<EOF
-; disable .user.ini files for performance and workaround NC update bugs
-user_ini.filename =
-
-; from Nextcloud .user.ini
-upload_max_filesize=10G
-post_max_size=10G
-memory_limit=768M
-mbstring.func_overload=0
-always_populate_raw_post_data=-1
-default_charset='UTF-8'
-output_buffering=0
-
-; slow transfers will be killed after this time
-max_execution_time=3600
-max_input_time=3600
-EOF
-
-# restart services
-service php${php_ver_new}-fpm restart
-a2enconf php${php_ver_new}-fpm
-service apache2 restart
-
-is_active_app unattended-upgrades && run_app unattended-upgrades || true
-
-# mark as successful
-mv "${new_cfg}" "${old_cfg}"
-install_template "php/opcache.ini.sh" "/etc/php/${php_ver_new}/mods-available/opcache.ini" --defaults
-clear_opcache
-
-source /usr/local/etc/library.sh # refresh NCPCFG RELEASE PHPVER
-run_app nc-limits
-restore_maintenance_mode
-
-rm -f /etc/update-motd.d/30ncp-dist-upgrade
-
-echo "Upgrade to ${release_new} successful"
-trap '' EXIT

bin/ncp-dist-upgrade.d/debian-10.sh

@@ -0,0 +1,137 @@
+#!/bin/bash
+
+set -eu -o pipefail
+
+[[ -f /.dockerenv ]] && { echo "Not supported in Docker. Upgrade the container instead"; exit 0; }
+
+new_cfg=/usr/local/etc/ncp-recommended.cfg
+[[ -f "${new_cfg}" ]] || { echo "Already on the lastest recommended distribution. Abort." >&2; exit 1; }
+
+APTINSTALL="apt-get install -y --no-install-recommends"
+export DEBIAN_FRONTEND=noninteractive
+
+echo "
+>>> ATTENTION <<<
+This is a dangerous process that is only guaranteed to work properly if you
+have not made manual changes in the system. Backup the SD card first and
+proceed at your own risk.
+
+Note that this is not a requirement for NCP to continue working properly.
+The current distribution will keep receiving updates for some time.
+
+Do you want to continue? [y/N]"
+
+read key
+[[ "$key" == y ]] || exit 0
+
+source /usr/local/etc/library.sh # sets NCPCFG RELEASE PHPVER
+old_cfg="${NCPCFG}"
+
+trap "echo 'Something went wrong. Fix it and try again'" EXIT
+
+save_maintenance_mode
+
+# Fix grub-pc issue in VM
+if apt show grub-pc-bin &>/dev/null; then
+  $APTINSTALL grub
+fi
+
+apt-get update
+apt-get upgrade -y
+
+# remove old PHP version
+set +e
+apt-get purge -y php${PHPVER} php${PHPVER}-curl php${PHPVER}-gd php${PHPVER}-fpm php${PHPVER}-cli php${PHPVER}-opcache \
+	php${PHPVER}-mbstring php${PHPVER}-xml php${PHPVER}-zip php${PHPVER}-fileinfo php${PHPVER}-ldap \
+	php${PHPVER}-intl php${PHPVER}-bz2 php${PHPVER}-json
+apt-get purge -y php${PHPVER}-mysql
+apt-get purge -y php${PHPVER}-redis
+apt-get purge -y php${PHPVER}-exif
+apt-get purge -y php${PHPVER}-bcmath
+apt-get purge -y php${PHPVER}-gmp
+apt-get purge -y php${PHPVER}-imagick
+set -e
+
+# update sources
+sed -i 's/buster/bullseye/g' /etc/apt/sources.list
+sed -i 's/buster/bullseye/g' /etc/apt/sources.list.d/* || true
+sed -i 's/bullseye\/updates/bullseye-security/g' /etc/apt/sources.list
+rm -f /etc/apt/sources.list.d/php.list
+
+# fix DHCP systemd service command https://forums.raspberrypi.com/viewtopic.php?t=320383 in raspbian
+if [[ -f /usr/bin/raspi-config ]]; then
+  sed -i 's|ExecStart=/usr/lib/dhcpcd5/dhcpcd -q -w|ExecStart=/usr/sbin/dhcpcd -q -w|g' /etc/systemd/system/dhcpcd.service.d/wait.conf
+fi
+
+# install latest distro
+apt-get update
+apt-get dist-upgrade -y
+
+# install latest PHP version
+release_new=$(jq -r '.release'     < "${new_cfg}")
+# the default repo in bullseye is bullseye-security - use bullseye if it is not available
+grep -Eh '^deb ' /etc/apt/sources.list | grep 'bullseye-security' > /dev/null && release_new="${release_new}-security"
+php_ver_new=$(jq -r '.php_version' < "${new_cfg}")
+# PHP 8.1 is only supported via the
+[[ "$php_ver_new" != 8.1 ]] || php_ver_new=7.4
+
+$APTINSTALL -t ${release_new} php${php_ver_new} php${php_ver_new}-curl php${php_ver_new}-gd php${php_ver_new}-fpm php${php_ver_new}-cli php${php_ver_new}-opcache \
+	php${php_ver_new}-mbstring php${php_ver_new}-xml php${php_ver_new}-zip php${php_ver_new}-fileinfo php${php_ver_new}-ldap \
+	php${php_ver_new}-intl php${php_ver_new}-bz2 php${php_ver_new}-json
+
+$APTINSTALL php${php_ver_new}-mysql
+$APTINSTALL -t ${release_new} php${php_ver_new}-redis
+
+$APTINSTALL -t ${release_new} smbclient exfat-fuse exfat-utils
+sleep 2 # avoid systemd thinking that PHP is in a crash/restart loop
+$APTINSTALL -t ${release_new} php${php_ver_new}-exif
+sleep 2 # avoid systemd thinking that PHP is in a crash/restart loop
+$APTINSTALL -t ${release_new} php${php_ver_new}-bcmath
+sleep 2 # avoid systemd thinking that PHP is in a crash/restart loop
+$APTINSTALL -t ${release_new} php${php_ver_new}-gmp
+#$APTINSTALL -t ${release_new} imagemagick php${php_ver_new}-imagick ghostscript
+
+# Reinstall prometheus-node-exporter, specifically WITH install-recommends to include collectors on bullseye and later
+{ dpkg -l | grep '^ii.*prometheus-node-exporter' >/dev/null && apt-get install -y prometheus-node-exporter-collectors; } || true
+
+apt-get autoremove -y
+apt-get clean
+
+cat > /etc/php/${php_ver_new}/fpm/conf.d/90-ncp.ini <<EOF
+; disable .user.ini files for performance and workaround NC update bugs
+user_ini.filename =
+
+; from Nextcloud .user.ini
+upload_max_filesize=10G
+post_max_size=10G
+memory_limit=768M
+mbstring.func_overload=0
+always_populate_raw_post_data=-1
+default_charset='UTF-8'
+output_buffering=0
+
+; slow transfers will be killed after this time
+max_execution_time=3600
+max_input_time=3600
+EOF
+
+# restart services
+service php${php_ver_new}-fpm restart
+a2enconf php${php_ver_new}-fpm
+service apache2 restart
+
+is_active_app unattended-upgrades && run_app unattended-upgrades || true
+
+# mark as successful
+mv "${new_cfg}" "${old_cfg}"
+install_template "php/opcache.ini.sh" "/etc/php/${php_ver_new}/mods-available/opcache.ini" --defaults
+clear_opcache
+
+source /usr/local/etc/library.sh # refresh NCPCFG RELEASE PHPVER
+run_app nc-limits
+restore_maintenance_mode
+
+rm -f /etc/update-motd.d/30ncp-dist-upgrade
+
+echo "Upgrade to ${release_new} successful"
+trap '' EXIT

bin/ncp-dist-upgrade.d/debian-11.sh

@@ -0,0 +1,86 @@
+    #!/bin/bash
+
+set -eu -o pipefail
+
+new_cfg=/usr/local/etc/ncp-recommended.cfg
+[[ -f "${new_cfg}" ]] || { echo "Already on the lastest recommended distribution. Abort." >&2; exit 1; }
+
+echo "
+>>> ATTENTION <<<
+This is a dangerous process that is only guaranteed to work properly if you
+have not made manual changes in the system. Backup the SD card first and
+proceed at your own risk.
+
+Note that this is not a requirement for NCP to continue working properly.
+The current distribution will keep receiving updates for some time.
+
+Do you want to continue? [y/N]"
+
+if [[ "${DEBIAN_FRONTEND:-}" == "noninteractive" ]] || ! [[ -t 0 ]]
+then
+  echo "Noninteractive environment detected. Automatically proceeding in 30 seconds..."
+  sleep 30
+else
+  read -n1 -r key
+  [[ "${key,,}" == y ]] || exit 0
+fi
+
+export DEBIAN_FRONTEND=noninteractive
+
+source /usr/local/etc/library.sh
+is_more_recent_than "${PHPVER}.0" "8.0.0" || {
+  echo "You still have PHP version ${PHPVER} installed. Please update to the latest supported version of nextcloud (which will also update your PHP version) before proceeding with the distribution upgrade."
+  echo "Exiting."
+  exit 1
+}
+save_maintenance_mode
+
+# Perform dist-upgrade
+
+apt-get update
+apt-get remove -y libc-dev-bin || true
+apt-get upgrade -y
+for aptlist in /etc/apt/sources.list /etc/apt/sources.list.d/{php.list,armbian.list,raspi.list}
+do
+  [ -f "$aptlist" ] && sed -i -e "s/bullseye/bookworm/g" "$aptlist"
+done
+for aptlist in /etc/apt/sources.list.d/*.list
+do
+	[[ "$aptlist" =~ "/etc/apt/sources.list.d/"(php|armbian|raspi)".list" ]] || {
+    echo "Disabling repositories from \"$aptlist\""
+    sed -i -e "s/deb/#deb/g" "$aptlist"
+  }
+done
+apt-get update
+apt-get upgrade -y dpkg
+apt-get upgrade -y --without-new-pkgs
+if is_lxc
+then
+  # Required to avoid breakage of /etc/resolv.conf
+  apt-get install -y --no-install-recommends systemd-resolved && systemctl enable --now systemd-resolved
+fi
+apt-get full-upgrade -y
+sudo apt-get install -y --no-install-recommends libc-dev-bin || true
+sudo apt-get --purge  autoremove -y
+
+apt-get install -y --no-install-recommends exfatprogs
+
+#mkdir -p /etc/systemd/system/php8.1-fpm.service.d
+#echo '[Service]' > /etc/systemd/system/php8.1-fpm.service.d/ncp.conf
+#echo 'ExecStartPre=mkdir -p /var/run/php' >> /etc/systemd/system/php8.1-fpm.service.d/ncp.conf
+#[[ "$INIT_SYSTEM" != "systemd" ]] || { systemctl daemon-reload && systemctl restart php8.1-fpm; }
+
+restore_maintenance_mode
+cfg="$(jq "." "$NCPCFG")"
+cfg="$(jq ".release = \"bookworm\"" <<<"$cfg")"
+echo "$cfg" > "$NCPCFG"
+rm -f /etc/update-motd.d/30ncp-dist-upgrade
+rm -f /usr/local/etc/ncp-recommended.cfg
+
+echo "Update to Debian 12 (bookworm) successful."
+
+is_active_app unattended-upgrades && {
+  echo "Setting up unattended upgrades..."
+  run_app unattended-upgrades || true
+  echo "done."
+}

bin/ncp-provisioning.sh

@@ -3,7 +3,7 @@
 # this script runs at startup to provide an unique random passwords for each instance
 
 source /usr/local/etc/library.sh
-
+set -x
 ## redis provisioning
 
 CFG=/var/www/nextcloud/config/config.php
@@ -67,9 +67,28 @@ BKP="$( ls -1t /var/www/nextcloud-bkp_*.tar.gz 2>/dev/null | head -1 )"
 ## Check for encrypted data and ask for password
 if needs_decrypt; then
   echo "Detected encrypted instance"
-  a2dissite ncp nextcloud
+  a2dissite ncp 001-nextcloud
   a2ensite ncp-activation
   apache2ctl -k graceful
 fi
 
+[[ -f /usr/local/etc/instance.cfg ]] || {
+  cohorte_id=$((RANDOM % 100))
+  cat > /usr/local/etc/instance.cfg <<EOF
+{
+  "cohorteId": ${cohorte_id}
+}
+EOF
+  cat /usr/local/etc/instance.cfg
+}
+
+systemctl is-enabled -q nextcloud-ai-worker@1.service || {
+  max="$(nproc || echo '2')"
+  max="$((max-1))"
+  for i in $(seq 1 "$max")
+  do
+    systemctl enable --now "nextcloud-ai-worker@${i}.service"
+  done
+}
+
 exit 0

bin/ncp-report

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# NextCloudPi diagnostics report
+# NextcloudPi diagnostics report
 #
 # Copyleft 2017 by Ignacio Nunez Hernanz <nacho _a_t_ ownyourbits _d_o_t_ com>
 # GPL licensed (see end of file) * Use at your own risk!

bin/ncp-restore

@@ -16,32 +16,32 @@ DIR="$( cd "$( dirname "$BACKUPFILE" )" &>/dev/null && pwd )" #abspath
 [[ -f "$BACKUPFILE"   ]] || { echo "$BACKUPFILE not found"          ; exit 1; }
 [[ "$DIR" =~ "$NCDIR".* ]] && { echo "Refusing to restore from $NCDIR"; exit 1; }
 
-TMPDIR="$( mktemp -d "$( dirname "$BACKUPFILE" )"/ncp-restore.XXXXXX )" || { echo "Failed to create temp dir" >&2; exit 1; }
-[[ "$(stat -fc%T "${TMPDIR}")" =~ ext|btrfs|zfs ]] || { echo "Can only restore from ext/btrfs/zfs filesystems (found '$(stat -fc%T "${TMPDIR}")"     >&2; exit 1; }
+TEMPDIR="$( mktemp -d "$( dirname "$BACKUPFILE" )"/ncp-restore.XXXXXX )" || { echo "Failed to create temp dir" >&2; exit 1; }
+[[ "$(stat -fc%T "${TEMPDIR}")" =~ ext|btrfs|zfs ]] || { echo "Can only restore from ext/btrfs/zfs filesystems (found '$(stat -fc%T "${TEMPDIR}")"     >&2; exit 1; }
 
-TMPDIR="$( cd "$TMPDIR" &>/dev/null && pwd )" || { echo "$TMPDIR not found"; exit 1; } #abspath
-cleanup(){  local RET=$?; echo "Cleanup..."; rm -rf "${TMPDIR}"; trap "" EXIT; exit $RET; }
+TEMPDIR="$( cd "$TEMPDIR" &>/dev/null && pwd )" || { echo "$TEMPDIR not found"; exit 1; } #abspath
+cleanup(){  local RET=$?; echo "Cleanup..."; rm -rf "${TEMPDIR}"; trap "" EXIT; exit $RET; }
 trap cleanup INT TERM HUP ERR EXIT
-rm -rf "$TMPDIR" && mkdir -p "$TMPDIR"
+rm -rf "$TEMPDIR" && mkdir -p "$TEMPDIR"
 
 [[ "$BACKUPFILE" =~ .*".tar.gz" ]] && compress_arg="-I pigz"
 
-# CHECK FREE SPACE IN $TMPDIR
+# CHECK FREE SPACE IN $TEMPDIR
 echo "check free space..." # allow at least ~100 extra MiB
 extractedsize=$(tar $compress_arg -tvf "$BACKUPFILE" | awk '{s+=$3} END{printf "%.0f", (s/1024)}') # Size of extracted files in "KB"
 size=$(($extractedsize + 100*1024))
-free=$( df "$TMPDIR" | tail -1 | awk '{ print $4 }' )
+free=$( df "$TEMPDIR" | tail -1 | awk '{ print $4 }' )
 [ $size -ge $free ] && {
-  echo "free space check failed. Need $size KB in $TMPDIR";
+  echo "free space check failed. Need $size KB in $TEMPDIR";
   exit 1;
 }
 
 # EXTRACT FILES
 echo "extracting backup file $BACKUPFILE..."
-tar $compress_arg -xf "$BACKUPFILE" -C "$TMPDIR" || exit 1
+tar $compress_arg -xf "$BACKUPFILE" -C "$TEMPDIR" || exit 1
 
 ## SANITY CHECKS
-[[ -d "$TMPDIR"/nextcloud ]] && [[ -f "$( ls "$TMPDIR"/nextcloud-sqlbkp_*.bak 2>/dev/null )" ]] || {
+[[ -d "$TEMPDIR"/nextcloud ]] && [[ -f "$( ls "$TEMPDIR"/nextcloud-sqlbkp_*.bak 2>/dev/null )" ]] || {
   echo "invalid backup file. Abort"
   exit 1
 }
@@ -54,7 +54,7 @@ echo "restore files..."
   mv -T "$NCDIR/data" "$DATA_BKP_DIR/"
 }
 rm -rf "$NCDIR"
-mv -T "$TMPDIR"/nextcloud "$NCDIR" || { echo "Error restoring base files"; exit 1; }
+mv -T "$TEMPDIR"/nextcloud "$NCDIR" || { echo "Error restoring base files"; exit 1; }
 
 if [[ -n "$DATA_BKP_DIR" ]]
 then
@@ -91,7 +91,7 @@ EOFMYSQL
 # shellcheck disable=SC2181
 [ $? -ne 0 ] && { echo "Error configuring nextcloud database"; exit 1; }
 
-mysql -u root nextcloud <  "$TMPDIR"/nextcloud-sqlbkp_*.bak || { echo "Error restoring nextcloud database"; exit 1; }
+mysql -u root nextcloud <  "$TEMPDIR"/nextcloud-sqlbkp_*.bak || { echo "Error restoring nextcloud database"; exit 1; }
 
 ## RESTORE DATADIR
 
@@ -105,7 +105,7 @@ cd "$NCDIR"
 
 ### INCLUDEDATA=yes situation
 NUMFILES=2
-if [[ $( ls "$TMPDIR" | wc -l ) -eq $NUMFILES ]]; then
+if [[ $( ls "$TEMPDIR" | wc -l ) -eq $NUMFILES ]]; then
 
   [[ -e "$DATADIR" ]] && {
     bk_target="$DATADIR-$( date '+%FT%s' )"
@@ -121,7 +121,7 @@ if [[ $( ls "$TMPDIR" | wc -l ) -eq $NUMFILES ]]; then
     btrfs subvolume create "$DATADIR" || exit 1
   }
   chown www-data: "$DATADIR"
-  TMPDATA="$TMPDIR/$( basename "$DATADIR" )"
+  TMPDATA="$TEMPDIR/$( basename "$DATADIR" )"
   mv "$TMPDATA"/* "$TMPDATA"/.[!.]* "$DATADIR" || exit 1
   rmdir "$TMPDATA"                             || exit 1
 

bin/ncp-suggestions

@@ -1,7 +1,7 @@
 #!/bin/bash
 
 #
-# NextCloudPi configuration suggestions
+# NextcloudPi configuration suggestions
 #
 # Copyleft 2018 by Ignacio Nunez Hernanz <nacho _a_t_ ownyourbits _d_o_t_ com>
 # GPL licensed (see end of file) * Use at your own risk!

bin/ncp-update

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# update latest NextCloudPi code from github
+# update latest NextcloudPi code from github
 
 [[ -z "$DBG" ]] || set -$DBG
 
@@ -19,18 +19,18 @@ fi
   BRANCH="${1:-master}"
   [[ "$BRANCH" != "master" ]] && echo "INFO: updating to development branch '$BRANCH'"
 
-  TMPDIR="$( mktemp -d /tmp/ncp-update.XXXXXX || ( echo "Failed to create temp dir. Exiting" >&2; exit 1 ) )"
-  trap 'cd /; rm -rf "${TMPDIR}"' EXIT
+  TEMPDIR="$( mktemp -d /tmp/ncp-update.XXXXXX || ( echo "Failed to create temp dir. Exiting" >&2; exit 1 ) )"
+  trap 'cd /; rm -rf "${TEMPDIR}"' EXIT
 
   echo -e "Downloading updates"
-  git clone --depth 20 -b "$BRANCH" -q https://github.com/nextcloud/nextcloudpi.git "$TMPDIR" || {
+  git clone --depth 20 -b "$BRANCH" -q https://github.com/nextcloud/nextcloudpi.git "$TEMPDIR" || {
     echo "No internet connectivity"
     exit 1
   }
 
   # shellcheck disable=SC2164
   [[ -f /.ncp-image ]] || {
-    cd "$TMPDIR"         # update locally during build
+    cd "$TEMPDIR"         # update locally during build
 
     [[ -z "$2" ]] || {
       git fetch origin "$2" || {
@@ -44,7 +44,7 @@ fi
   echo -e "Performing updates"
   ./update.sh || exit $?
 
-  cd "$TMPDIR" || exit 1
+  cd "$TEMPDIR" || exit 1
   git describe --always --tags --abbrev=0 2>/dev/null | grep -qoP "v\d+\.\d+\.\d+" || git fetch --unshallow --tags -q
   VER=$( git describe --always --tags --abbrev=0 | grep -oP "v\d+\.\d+\.\d+" )
 
@@ -61,7 +61,7 @@ fi
     sed '/HEAD ->\|origin/s|\[.*\(tag: v[0-9]\+\.[0-9]\+\.[0-9]\+\).*\]|[\1]|' | \
     sed 's|* \[tag: |[|' > /usr/local/etc/ncp-changelog
 
-  echo -e "NextCloudPi updated to version $VER"
+  echo -e "NextcloudPi updated to version $VER"
   exit 0
 
 } # force to read the whole thing into memory, as its contents might change in update.sh

bin/ncp-update-nc

@@ -19,297 +19,50 @@
 
 set -eE${DBG}
 
-VER="$1"
 BIN="${0##*/}"
-
-source /usr/local/etc/library.sh
-
-# pre-checks
-####################
+VER="$1"
 [[ "$VER" == "" ]] && { echo "Usage ${BIN} <version>"; exit 1; }
-[[ -f /.docker-image ]] && BASEDIR=/data || BASEDIR=/var/www
-cd "$BASEDIR"
-DATADIR="$( get_nc_config_value datadirectory )"
-ncc status &>/dev/null          || { [[ "$DBG" == x ]] && ncc status; echo "Nextcloud is currently down"; exit 1; }
-[[ -d "${BASEDIR}/nextcloud-old" ]] && { echo "Nextcloud backup directory found. Interrupted or already running installation?"; exit 1; }
-[[ -d "${BASEDIR}/nextcloud"     ]] || { echo "Nextcloud directory not found"      ; exit 1; }
-[[ -d "$DATADIR"             ]] || { echo "Nextcloud data directory not found" ; exit 1; }
 
-# check version
-####################
+connect_to_nc_update() {
+  tail -n 100 -f "/var/log/ncp-update-nc.log" &
+  tail_pid=$!
+  trap "kill '$tail_pid'" EXIT
+  while [[ "$(systemctl is-active ncp-update-nc ||:)" =~ ^(active|activating|deactivating)$ ]]
+  do
+    sleep 3
+  done
 
-[[ ${EUID} -eq 0 ]] && SUDO="sudo -u www-data"
-CURRENT="$( $SUDO php /var/www/nextcloud/occ status | grep "version:" | awk '{ print $3 }' )"
-
-MAJOR_CUR=$( cut -d. -f1 <<<"${CURRENT}" )
-MAJOR_NEW=$( cut -d. -f1 <<<"${VER}" )
-if [[ $((MAJOR_NEW - MAJOR_CUR)) -gt 1 ]]; then
-  echo "Upgrade cannot skip major versions. Please upgrade one major version at a time" >&2
-  exit 1
-fi
-
-if [[ "$MAJOR_NEW" -ge 24 ]] && [[ "$(lsb_release -r)" =~ .*10 ]]
-then
-  echo -e "NCP doesn't support Nextcloud versions greater than 23 with Debian 10 (Buster). Please run ncp-dist-upgrade."
-  exit 1
-fi
-
-grep -qP "\d+\.\d+\.\d+" <<<"$CURRENT" || { echo "Malformed version $CURRENT"; exit 1; }
-grep -qP "\d+\.\d+\.\d+" <<<"$VER"     || { echo "Malformed version $VER"    ; exit 1; }
-
-echo "Current   Nextcloud version $CURRENT"
-echo "Available Nextcloud version $VER"
-is_more_recent_than "${VER}" "${CURRENT}" || { echo "Nothing to update"; exit 1; } # we want `exit 1` so the autoupdate doesn't notify success in this case
-
-
-if ! is_more_recent_than '25.0.0' "${VER}" && is_more_recent_than "8.1.0" "${PHPVER}.0" && is_docker
-then
-  echo 'You need to upgrade to a later docker image in order to upgrade to Nextcloud 25+'
-  exit 1
-fi
-
-# make sure that cron.php is not running and there are no pending jobs
-# https://github.com/nextcloud/server/issues/10949
-pgrep -cf cron.php &>/dev/null && { pkill -f cron.php; sleep 3; }
-pgrep -cf cron.php &>/dev/null && { echo "cron.php running. Abort"; exit 1; }
-mysql nextcloud <<<"UPDATE oc_jobs SET reserved_at=0;"
-
-# cleanup
-####################
-cleanup() {
-  local RET=$?
-  set +eE
-  echo "Clean up..."
-  rm -rf "$BASEDIR"/nextcloud.tar.bz2 "$BASEDIR"/nextcloud-old
-  trap "" EXIT
-  exit $RET
-}
-trap cleanup EXIT
-
-# get new code
-####################
-URL="https://download.nextcloud.com/server/releases/nextcloud-$VER.tar.bz2"
-echo "Download Nextcloud $VER..."
-wget -q "$URL" -O nextcloud.tar.bz2 || { echo "Error downloading"; exit 1; }
-
-# backup
-####################
-BKPDIR="$BASEDIR"
-WITH_DATA=no
-COMPRESSED=yes
-LIMIT=0
-
-echo "Back up current instance..."
-set +eE
-ncp-backup "$BKPDIR" "$WITH_DATA" "$COMPRESSED" "$LIMIT" # && false # test point
-RET=$?
-sync
-set -eE
-
-BKP_="$( ls -1t "$BKPDIR"/nextcloud-bkp_*.tar.gz 2>/dev/null | head -1 )"
-[[ -f "$BKP_"  ]] || {                set +eE; echo "Error backing up"; false || cleanup; }
-[[ $RET -ne 0  ]] && { rm -f "$BKP_"; set +eE; echo "Error backing up"; false || cleanup; }
-BKP="$( dirname "$BKP_" )/$( basename "$BKP_" .tar.gz )-${CURRENT}.tar.gz"
-echo "Storing backup at '$BKP'..."
-mv "$BKP_" "$BKP"
-
-# simple restore if anything fails from here
-####################
-rollback_simple() {
-  set +eE
-  trap "" INT TERM HUP ERR
-  echo -e "Abort\nSimple roll back..."
-  rm -rf "$BASEDIR"/nextcloud
-  mv "$BASEDIR"/nextcloud-old "$BASEDIR"/nextcloud
-  false || cleanup                 # so cleanup exits with 1
-}
-trap rollback_simple INT TERM HUP ERR
-
-# replace code
-####################
-echo "Install Nextcloud $VER..."
-mv -T nextcloud nextcloud-old
-tar -xf nextcloud.tar.bz2             # && false # test point
-rm -rf /var/www/nextcloud.tar.bz2
-
-# copy old config
-####################
-cp nextcloud-old/config/config.php nextcloud/config/
-
-# copy old themes
-####################
-cp -raT nextcloud-old/themes/ nextcloud/themes/
-
-# copy old NCP apps
-####################
-for app in nextcloudpi previewgenerator; do
-  if [[ -d nextcloud-old/apps/"${app}" ]]; then
-    cp -r -L nextcloud-old/apps/"${app}" /var/www/nextcloud/apps/
+  if [[ "$(systemctl is-active ncp-update-nc ||:)" == "inactive" ]]
+  then
+    echo "Nextcloud update finished successfully."
+    return 0
+  elif [[ "$(systemctl is-active ncp-update-nc ||:)" == "failed" ]]
+  then
+    echo "Nextcloud update failed (or was installed already)."
+    return 1
+  else
+    echo "Nextcloud update was not found or failed (unexpected status: '$(systemctl is-active ncp-update-nc ||:)')"
   fi
-done
-
-#false # test point
-
-# copy data if it was at the default location
-####################
-if [[ "$DATADIR" == "/var/www/nextcloud/data" ]] || [[ "$DATADIR" == "/data/nextcloud/data" ]]; then
-  echo "Restore data..."
-  mv -T nextcloud-old/data nextcloud/data
-fi
-
-# nc-restore if anything fails from here
-####################
-rollback() {
-  set +eE
-  trap "" INT TERM HUP ERR EXIT
-  echo -e "Abort\nClean up..."
-  rm -rf /var/www/nextcloud.tar.bz2 "$BASEDIR"/nextcloud-old
-  echo "Rolling back to backup $BKP..."
-  local TMPDATA
-  mkdir -p "$BASEDIR/recovery/"
-  TMPDATA="$( mktemp -d "$BASEDIR/recovery/ncp-data.XXXXXX" )" || { echo "Failed to create temp dir" >&2; exit 1; }
-  [[ "$DATADIR" == "$BASEDIR/nextcloud/data" ]] && mv -T "$DATADIR" "$TMPDATA"
-  ncp-restore "$BKP" || { echo "Rollback failed! Data left at $TMPDATA"; exit 1; }
-  [[ "$DATADIR" == "$BASEDIR/nextcloud/data" ]] && { rm -rf "$DATADIR"; mv -T "$TMPDATA" "$DATADIR"; }
-  rm "$BKP"
-  echo "Rollback successful. Nothing was updated"
-  exit 1
 }
-trap rollback INT TERM HUP ERR
 
-# fix permissions
-####################
-echo "Fix permissions..."
-chown -R www-data:www-data nextcloud
-find nextcloud/ -type d -exec chmod 750 {} \;
-find nextcloud/ -type f -exec chmod 640 {} \;
-
-# upgrade
-####################
-echo "Upgrade..."
-ncc='sudo -u www-data php nextcloud/occ'
-$ncc upgrade      # && false # test point
-$ncc | grep -q db:add-missing-indices && $ncc db:add-missing-indices -n
-$ncc | grep -q db:add-missing-columns && $ncc db:add-missing-columns -n
-$ncc | grep -q db:add-missing-primary-keys && $ncc db:add-missing-primary-keys -n
-$ncc | grep -q db:convert-filecache-bigint && $ncc db:convert-filecache-bigint -n
-
-# use the correct version for custom apps
-NCVER="$(ncc status | grep "version:" | awk '{ print $3 }')"
-if is_more_recent_than "21.0.0" "${NCVER}"; then
-  NCPREV=/var/www/ncp-previewgenerator/ncp-previewgenerator-nc20
-else
-  # Install notify_push if not installed
-  if ! is_app_enabled notify_push; then
-    ncc app:install notify_push
-    ncc app:enable  notify_push
-    install_template nextcloud.conf.sh /etc/apache2/sites-available/nextcloud.conf
-    a2enmod proxy proxy_http proxy_wstunnel
-    apachectl -k graceful
-    ## make sure the notify_push daemon is runnnig
-
-    install_template systemd/notify_push.service.sh /etc/systemd/system/notify_push.service
-    start_notify_push
-    nc_domain="$(ncc config:system:get overwrite.cli.url)"
-    set-nc-domain "${nc_domain}" || {
-      echo "notify_push setup failed. You are probably behind a proxy"
-      echo "Run 'ncc config:system:set trusted_proxies 15 --value=<proxy_IP>' and then 'ncc notify_push:setup https://<domain>/push to enable"
-      echo "Check https://help.nextcloud.com/tags/ncp for support"
-    }
-
-  fi
-  NCPREV=/var/www/ncp-previewgenerator/ncp-previewgenerator-nc21
-fi
-rm -rf /var/www/nextcloud/apps/previewgenerator
-ln -snf "${NCPREV}" /var/www/nextcloud/apps/previewgenerator
-
-if ! is_docker && ! is_more_recent_than "24.0.0" "${NCVER}" && is_more_recent_than "8.1.0" "${PHPVER}.0"
+if [[ "$(systemctl is-active ncp-update-nc ||:)" =~ ^(active|activating|deactivating)$ ]]
 then
-  (
-  echo "Upgrading PHP..."
-  export DEBIAN_FRONTEND=noninteractive
-  PHPVER_OLD="$PHPVER"
-  PHPVER_NEW="8.1"
-  PHP_PACKAGES_OLD=(php-{common,igbinary,redis,json} "php${PHPVER_OLD}" \
-    "php${PHPVER_OLD}"-{curl,gd,fpm,cli,opcache,mbstring,xml,zip,fileinfo,ldap,intl,bz2,json,common,readline,mysql,bcmath,gmp})
-  PHP_PACKAGES_NEW=("php${PHPVER_NEW}" \
-    "php${PHPVER_NEW}"-{curl,gd,fpm,cli,opcache,mbstring,xml,zip,fileinfo,ldap,intl,bz2,mysql,bcmath,gmp,redis,common})
-
-  php_restore() {
-    trap "" INT TERM HUP ERR
-    echo "Something went wrong while upgrading PHP. Rolling back to version ${PHPVER_OLD}..."
-    set +e
-    service "php${PHPVER_NEW}-fpm" stop
-    a2disconf php${PHPVER_NEW}-fpm
-    rm /etc/apt/sources.list.d/php.list
-    apt-get update
-    apt-get remove --purge -y "${PHP_PACKAGES_NEW[@]}"
-    apt-get install -y --no-install-recommends -t "$RELEASE" "${PHP_PACKAGES_OLD[@]}"
-    set_ncpcfg "php_version" "${PHPVER_OLD}"
-    install_template "php/opcache.ini.sh" "/etc/php/${PHPVER_NEW}/mods-available/opcache.ini"
-    run_app nc-limits
-    a2enconf "php${PHPVER_OLD}-fpm"
-    service "php${PHPVER_OLD}-fpm" start
-    service apache2 restart
-    echo "PHP upgrade has been successfully reverted"
-    set -e
-  }
-
-  trap php_restore INT TERM HUP ERR
-
-  # Setup apt repository for php 8
-  wget -O /etc/apt/trusted.gpg.d/php.gpg https://packages.sury.org/php/apt.gpg
-  echo "deb https://packages.sury.org/php/ ${RELEASE%-security} main" > /etc/apt/sources.list.d/php.list
-  apt-get update
-
-  clear_opcache
-
-  echo "Stopping apache and  php-fpm..."
-  service "php${PHPVER_OLD}-fpm" stop
-  service apache2 stop
-
-  echo "Remove old PHP (${PHPVER_OLD})..."
-  a2disconf "php${PHPVER_OLD}-fpm"
-
-  apt-get remove --purge -y "${PHP_PACKAGES_OLD[@]}"
-
-  echo "Install PHP ${PHPVER_NEW}..."
-  install_with_shadow_workaround --no-install-recommends systemd
-  apt-get install -y --no-install-recommends -t "$RELEASE" "${PHP_PACKAGES_NEW[@]}"
-
-  set_ncpcfg "php_version" "${PHPVER_NEW}"
-  install_template "php/opcache.ini.sh" "/etc/php/${PHPVER_NEW}/mods-available/opcache.ini"
-  ( set -e; export PHPVER="${PHPVER_NEW}"; run_app nc-limits )
-
-  a2enconf "php${PHPVER_NEW}-fpm"
-
-  echo "Starting apache and php-fpm..."
-  service "php${PHPVER_NEW}-fpm" start
-  service apache2 start
-  ncc status
-  )
-
-  # Reload library.sh to reset PHPVER
-  source /usr/local/etc/library.sh
-
+  echo "Existing ncp-update-nc process detected. Connecting..."
+  connect_to_nc_update
+  exit $?
 fi
 
-# refresh completions
-ncc _completion -g --shell-type bash -p ncc | sed 's|/var/www/nextcloud/occ|ncc|g' > /usr/share/bash-completion/completions/ncp
+systemctl reset-failed ncp-update-nc 2>/dev/null ||:
+systemd-run -u 'ncp-update-nc' --service-type=oneshot --no-block -p TimeoutStartSec="24h" -p TimeoutStopSec="1h" \
+  bash -c "set -o pipefail; DBG='${DBG:-}' /usr/local/bin/ncp-update-nc.d/update-nc.sh '${VER}' |& tee /var/log/ncp-update-nc.log"
+sleep 1
 
-is_docker && {
-  killall notify_push
-  sleep 1
-  start_notify_push
-}
+if ! [[ "$(systemctl is-active ncp-update-nc ||:)" =~ ^(active|inactive|activating|deactivating)$ ]]
+then
+  echo "Failed to start ncp-update-nc"
+  [[ -f /var/log/ncp-update-nc.log ]] && cat /var/log/ncp-update-nc.log
+  systemctl status --no-pager ncp-update-nc ||:
+  exit 1
+fi
 
-echo "Update completed successfully."
-# done
-####################
-mkdir -p  "$DATADIR"/ncp-update-backups
-mv "$BKP" "$DATADIR"/ncp-update-backups
-chown -R www-data:www-data "$DATADIR"/ncp-update-backups
-BKP="$DATADIR"/ncp-update-backups/"$( basename "$BKP" )"
-echo "Backup stored at $BKP"
-
-bash -c "sleep 5; source /usr/local/etc/library.sh; clear_opcache;" &>/dev/null &
+connect_to_nc_update

bin/ncp-update-nc.d/update-nc.sh

@@ -0,0 +1,261 @@
+#!/bin/bash
+
+set -eE${DBG}
+
+VER="$1"
+source /usr/local/etc/library.sh
+export RELEASE
+export PHPVER
+
+# pre-checks
+####################
+BASEDIR=/var/www
+cd "$BASEDIR"
+DATADIR="$( get_nc_config_value datadirectory )"
+ncc status &>/dev/null          || { [[ "$DBG" == x ]] && ncc status; echo "Nextcloud is currently down"; exit 1; }
+[[ -d "${BASEDIR}/nextcloud-old" ]] && { echo "Nextcloud backup directory found. Interrupted or already running installation?"; exit 1; }
+[[ -d "${BASEDIR}/nextcloud"     ]] || { echo "Nextcloud directory not found"      ; exit 1; }
+[[ -d "$DATADIR"             ]] || { echo "Nextcloud data directory not found" ; exit 1; }
+
+# check version
+####################
+
+[[ ${EUID} -eq 0 ]] && SUDO="sudo -u www-data"
+CURRENT="$(nc_version)"
+if [[ "$VER" == "0" ]] || [[ "$VER" == "" ]]
+then
+  REQUESTED_VERSION="latest"
+else
+  REQUESTED_VERSION="$VER"
+fi
+TARGET_VERSION="$(determine_nc_update_version "${CURRENT?}" "${NCLATESTVER}" "${REQUESTED_VERSION}")"
+[[ "$TARGET_VERSION" == "$CURRENT" ]] && {
+  echo "Nextcloud version ${CURRENT} is already installed. Nothing to do."
+  exit 1
+}
+[[ -n "$TARGET_VERSION" ]] || {
+  echo "Could not find a valid upgrade path from '${CURRENT}' to '${TARGET_VERSION}'. Nothing to update."
+  exit 1
+}
+
+MAJOR_NEW="${TARGET_VERSION%%.*}"
+DEBIAN_VERSION="$(. /etc/os-release; echo "$VERSION_ID")"
+
+if [[ "$MAJOR_NEW" -ge 24 ]] && [[ $DEBIAN_VERSION -le 10 ]]
+then
+  echo -e "NCP doesn't support Nextcloud versions greater than 23 with Debian 10 (Buster). Please run ncp-dist-upgrade."
+  exit 1
+fi
+
+if [[ "$MAJOR_NEW" -ge 29 ]] && [[ $DEBIAN_VERSION -le 11 ]]
+then
+  echo -e "NCP doesn't support Nextcloud versions greater than 28 with Debian 11 (Bullseye). Please run ncp-dist-upgrade."
+  exit 1
+fi
+
+grep -qP "\d+\.\d+\.\d+" <<<"$CURRENT" || { echo "Malformed version $CURRENT"; exit 1; }
+grep -qP "\d+\.\d+\.\d+" <<<"$TARGET_VERSION"   || { echo "Malformed version $TARGET_VERSION"    ; exit 1; }
+
+echo "Current   Nextcloud version $CURRENT"
+echo "Requested Nextcloud version $REQUESTED_VERSION"
+echo "Selected  Nextcloud version $TARGET_VERSION"
+
+if [[ "$REQUESTED_VERSION" == "latest" ]]
+then
+  echo "INFO: You have requested an update to the latest available version that can be performed directly, which is '${TARGET_VERSION}'. Updates can only performed to the next major version (or the latest minor version of the current major version). If you run ncp-update-nc again after this update completes, a new version might be available."
+  sleep 3
+elif [[ "$TARGET_VERSION" != "$REQUESTED_VERSION" ]]
+then
+  echo "INFO: You have requested an update to '${REQUESTED_VERSION}', but a direct update to '${REQUESTED_VERSION}' cannot be performed, so the latest available version that can be updated to (${TARGET_VERSION}) has been selected automatically."
+  sleep 3
+fi
+
+# make sure that cron.php is not running and there are no pending jobs
+# https://github.com/nextcloud/server/issues/10949
+pgrep -cf cron.php &>/dev/null && { pkill -f cron.php; sleep 3; }
+pgrep -cf cron.php &>/dev/null && { echo "cron.php running. Abort"; exit 1; }
+mysql nextcloud <<<"UPDATE ${DB_PREFIX}jobs SET reserved_at=0;"
+
+# cleanup
+####################
+cleanup() {
+  local RET=$?
+  set +eE
+  echo "Clean up..."
+  rm -rf "$BASEDIR"/nextcloud.tar.bz2 "$BASEDIR"/nextcloud-old
+  trap "" EXIT
+  exit $RET
+}
+trap cleanup EXIT
+
+# get new code
+####################
+URL="https://download.nextcloud.com/server/releases/nextcloud-$TARGET_VERSION.tar.bz2"
+echo "Download Nextcloud $TARGET_VERSION..."
+wget -q "$URL" -O nextcloud.tar.bz2 || { echo "Error downloading"; exit 1; }
+
+# backup
+####################
+BKPDIR="$BASEDIR"
+WITH_DATA=no
+COMPRESSED=yes
+LIMIT=0
+
+echo "Back up current instance..."
+set +eE
+ncp-backup "$BKPDIR" "$WITH_DATA" "$COMPRESSED" "$LIMIT" # && false # test point
+RET=$?
+sync
+set -eE
+
+BKP_="$( ls -1t "$BKPDIR"/nextcloud-bkp_*.tar.gz 2>/dev/null | head -1 )"
+[[ -f "$BKP_"  ]] || {                set +eE; echo "Error backing up"; false || cleanup; }
+[[ $RET -ne 0  ]] && { rm -f "$BKP_"; set +eE; echo "Error backing up"; false || cleanup; }
+BKP="$( dirname "$BKP_" )/$( basename "$BKP_" .tar.gz )-${CURRENT}.tar.gz"
+echo "Storing backup at '$BKP'..."
+mv "$BKP_" "$BKP"
+
+# simple restore if anything fails from here
+####################
+rollback_simple() {
+  set +eE
+  trap "" INT TERM HUP ERR
+  echo -e "Abort\nSimple roll back..."
+  rm -rf "$BASEDIR"/nextcloud
+  mv "$BASEDIR"/nextcloud-old "$BASEDIR"/nextcloud
+  false || cleanup                 # so cleanup exits with 1
+}
+trap rollback_simple INT TERM HUP ERR
+
+# replace code
+####################
+echo "Install Nextcloud $TARGET_VERSION..."
+mv -T nextcloud nextcloud-old
+tar -xf nextcloud.tar.bz2             # && false # test point
+rm -rf /var/www/nextcloud.tar.bz2
+
+# copy old config
+####################
+cp nextcloud-old/config/config.php nextcloud/config/
+
+# copy old themes
+####################
+cp -raT nextcloud-old/themes/ nextcloud/themes/
+
+# copy old NC apps
+####################
+for app in nextcloud-old/apps/*; do
+  if ! [[ -d /var/www/nextcloud/apps/"$(basename "$app")" ]]
+  then
+    cp -r -L "${app}" /var/www/nextcloud/apps/
+  fi
+done
+
+#false # test point
+
+# copy data if it was at the default location
+####################
+if [[ "$DATADIR" == "/var/www/nextcloud/data" ]] || [[ "$DATADIR" == "/data/nextcloud/data" ]]; then
+  echo "Restore data..."
+  mv -T nextcloud-old/data nextcloud/data
+fi
+
+# nc-restore if anything fails from here
+####################
+rollback() {
+  set +eE
+  trap "" INT TERM HUP ERR EXIT
+  echo -e "Abort\nClean up..."
+  rm -rf /var/www/nextcloud.tar.bz2 "$BASEDIR"/nextcloud-old
+  echo "Rolling back to backup $BKP..."
+  local TMPDATA
+  mkdir -p "$BASEDIR/recovery/"
+  TMPDATA="$( mktemp -d "$BASEDIR/recovery/ncp-data.XXXXXX" )" || { echo "Failed to create temp dir" >&2; exit 1; }
+  [[ "$DATADIR" == "$BASEDIR/nextcloud/data" ]] && mv -T "$DATADIR" "$TMPDATA"
+  ncp-restore "$BKP" || { echo "Rollback failed! Data left at $TMPDATA"; exit 1; }
+  [[ "$DATADIR" == "$BASEDIR/nextcloud/data" ]] && { rm -rf "$DATADIR"; mv -T "$TMPDATA" "$DATADIR"; }
+  rm "$BKP"
+  echo "Rollback successful. Nothing was updated"
+  exit 1
+}
+trap rollback INT TERM HUP ERR
+
+# fix permissions
+####################
+echo "Fix permissions..."
+chown -R www-data:www-data nextcloud
+find nextcloud/ -type d -exec chmod 750 {} \;
+find nextcloud/ -type f -exec chmod 640 {} \;
+
+# upgrade
+####################
+echo "Upgrade..."
+ncc='sudo -u www-data php nextcloud/occ'
+$ncc upgrade      # && false # test point
+$ncc | grep -q db:add-missing-indices && $ncc db:add-missing-indices -n
+$ncc | grep -q db:add-missing-columns && $ncc db:add-missing-columns -n
+$ncc | grep -q db:add-missing-primary-keys && $ncc db:add-missing-primary-keys -n
+$ncc | grep -q db:convert-filecache-bigint && $ncc db:convert-filecache-bigint -n
+$ncc | grep -q db:convert-mysql-charset && $ncc db:convert-mysql-charset -n
+$ncc maintenance:repair --help | grep -q -e '--include-expensive' && $ncc maintenance:repair --include-expensive
+if $ncc app_api:daemon:list | grep 'No registered daemon configs.' > /dev/null 2>&1
+then
+  $ncc app:disable app_api
+fi
+
+# use the correct version for custom apps
+NCVER="$(nc_version)"
+if is_more_recent_than "21.0.0" "${NCVER}"; then
+  NCPREV=/var/www/ncp-previewgenerator/ncp-previewgenerator-nc20
+else
+  # Install notify_push if not installed
+  if ! is_app_enabled notify_push; then
+    ncc app:install notify_push
+    ncc app:enable  notify_push
+    install_template nextcloud.conf.sh /etc/apache2/sites-available/nextcloud.conf
+    a2enmod proxy proxy_http proxy_wstunnel
+    apachectl -k graceful
+    ## make sure the notify_push daemon is runnnig
+
+    install_template systemd/notify_push.service.sh /etc/systemd/system/notify_push.service
+    start_notify_push
+    nc_domain="$(ncc config:system:get overwrite.cli.url)"
+    set-nc-domain "${nc_domain}" || {
+      echo "notify_push setup failed. You are probably behind a proxy"
+      echo "Run 'ncc config:system:set trusted_proxies 15 --value=<proxy_IP>' and then 'ncc notify_push:setup https://<domain>/push to enable"
+      echo "Check https://help.nextcloud.com/tags/ncp for support"
+    }
+
+  fi
+  NCPREV=/var/www/ncp-previewgenerator/ncp-previewgenerator-nc21
+fi
+rm -rf /var/www/nextcloud/apps/previewgenerator
+ln -snf "${NCPREV}" /var/www/nextcloud/apps/previewgenerator
+
+if ! is_more_recent_than "24.0.0" "${NCVER}" && is_more_recent_than "8.1.0" "${PHPVER}.0"
+then
+  /usr/local/bin/ncp-update-nc.d/upgrade-php-bullseye-8.1.sh
+
+  # Reload library.sh to reset PHPVER
+  source /usr/local/etc/library.sh
+elif ! is_more_recent_than "29.0.0" "${NCVER}" && is_more_recent_than "8.3.0" "${PHPVER}.0" && [[ "$DEBIAN_VERSION" -ge 12 ]]
+then
+  /usr/local/bin/ncp-update-nc.d/upgrade-php-bookworm-8.3.sh
+
+  # Reload library.sh to reset PHPVER
+  source /usr/local/etc/library.sh
+fi
+
+# refresh completions
+ncc _completion -g --shell-type bash -p ncc | sed 's|/var/www/nextcloud/occ|ncc|g' > /usr/share/bash-completion/completions/ncp
+
+echo "Update completed successfully."
+# done
+####################
+mkdir -p  "$DATADIR"/ncp-update-backups
+mv "$BKP" "$DATADIR"/ncp-update-backups
+chown -R www-data:www-data "$DATADIR"/ncp-update-backups
+BKP="$DATADIR"/ncp-update-backups/"$( basename "$BKP" )"
+echo "Backup stored at $BKP"
+
+bash -c "sleep 5; source /usr/local/etc/library.sh; clear_opcache;" &>/dev/null &

bin/ncp-update-nc.d/upgrade-php-bookworm-8.3.sh

@@ -0,0 +1,68 @@
+#!/usr/bin/env bash
+
+source /usr/local/etc/library.sh
+
+echo "Upgrading PHP..."
+export DEBIAN_FRONTEND=noninteractive
+PHPVER_OLD="$PHPVER"
+PHPVER_NEW="8.3"
+PHP_PACKAGES_OLD=("php${PHPVER_OLD}" \
+  "php${PHPVER_OLD}"-{curl,gd,fpm,cli,opcache,mbstring,xml,zip,fileinfo,ldap,intl,bz2,mysql,bcmath,gmp,redis,common})
+PHP_PACKAGES_NEW=("php${PHPVER_NEW}" \
+  "php${PHPVER_NEW}"-{curl,gd,fpm,cli,opcache,mbstring,xml,zip,fileinfo,ldap,intl,bz2,mysql,bcmath,gmp,redis,common})
+
+php_restore() {
+  trap "" INT TERM HUP ERR
+  echo "Something went wrong while upgrading PHP. Rolling back to version ${PHPVER_OLD}..."
+  set +e
+  service "php${PHPVER_NEW}-fpm" stop
+  a2disconf php${PHPVER_NEW}-fpm
+  wget -O /etc/apt/trusted.gpg.d/php.gpg https://packages.sury.org/php/apt.gpg
+  echo "deb https://packages.sury.org/php/ ${RELEASE%-security} main" > /etc/apt/sources.list.d/php.list
+  apt-get update
+  apt-get remove --purge -y "${PHP_PACKAGES_NEW[@]}"
+  apt-get install -y --no-install-recommends -t "$RELEASE" "${PHP_PACKAGES_OLD[@]}"
+  set_ncpcfg "php_version" "${PHPVER_OLD}"
+  install_template "php/opcache.ini.sh" "/etc/php/${PHPVER_NEW}/mods-available/opcache.ini"
+  run_app nc-limits
+  a2enconf "php${PHPVER_OLD}-fpm"
+  service "php${PHPVER_OLD}-fpm" start
+  service apache2 restart
+  echo "PHP upgrade has been successfully reverted"
+  set -e
+}
+
+trap php_restore INT TERM HUP ERR
+
+apt-get update
+
+clear_opcache
+
+echo "Stopping apache and  php-fpm..."
+service "php${PHPVER_OLD}-fpm" stop
+service apache2 stop
+
+echo "Remove old PHP (${PHPVER_OLD})..."
+a2disconf "php${PHPVER_OLD}-fpm"
+
+apt-get remove --purge -y "${PHP_PACKAGES_OLD[@]}"
+
+echo "Install PHP ${PHPVER_NEW}..."
+install_with_shadow_workaround --no-install-recommends systemd
+apt-get install -y --no-install-recommends -t "$RELEASE" "${PHP_PACKAGES_NEW[@]}"
+
+set_ncpcfg "php_version" "${PHPVER_NEW}"
+install_template "php/opcache.ini.sh" "/etc/php/${PHPVER_NEW}/mods-available/opcache.ini"
+( set -e; export PHPVER="${PHPVER_NEW}"; run_app nc-limits )
+
+a2enconf "php${PHPVER_NEW}-fpm"
+
+[[ -f "/etc/systemd/system/php${PHPVER_OLD}-fpm.service.d/ncp.conf" ]] && {
+  mkdir -p "/etc/systemd/system/php${PHPVER_NEW}-fpm.service.d"
+  cp "/etc/systemd/system/php${PHPVER_OLD}-fpm.service.d/ncp.conf" "/etc/systemd/system/php${PHPVER_NEW}-fpm.service.d/ncp.conf"
+}
+
+echo "Starting apache and php-fpm..."
+service "php${PHPVER_NEW}-fpm" start
+service apache2 start
+ncc status

bin/ncp-update-nc.d/upgrade-php-bullseye-8.1.sh

@@ -0,0 +1,65 @@
+#!/usr/bin/env bash
+
+source /usr/local/etc/library.sh
+
+echo "Upgrading PHP..."
+export DEBIAN_FRONTEND=noninteractive
+PHPVER_OLD="$PHPVER"
+PHPVER_NEW="8.1"
+PHP_PACKAGES_OLD=(php-{common,igbinary,redis,json} "php${PHPVER_OLD}" \
+  "php${PHPVER_OLD}"-{curl,gd,fpm,cli,opcache,mbstring,xml,zip,fileinfo,ldap,intl,bz2,json,common,readline,mysql,bcmath,gmp})
+PHP_PACKAGES_NEW=("php${PHPVER_NEW}" \
+  "php${PHPVER_NEW}"-{curl,gd,fpm,cli,opcache,mbstring,xml,zip,fileinfo,ldap,intl,bz2,mysql,bcmath,gmp,redis,common})
+
+php_restore() {
+  trap "" INT TERM HUP ERR
+  echo "Something went wrong while upgrading PHP. Rolling back to version ${PHPVER_OLD}..."
+  set +e
+  service "php${PHPVER_NEW}-fpm" stop
+  a2disconf php${PHPVER_NEW}-fpm
+  rm /etc/apt/sources.list.d/php.list
+  apt-get update
+  apt-get remove --purge -y "${PHP_PACKAGES_NEW[@]}"
+  apt-get install -y --no-install-recommends -t "$RELEASE" "${PHP_PACKAGES_OLD[@]}"
+  set_ncpcfg "php_version" "${PHPVER_OLD}"
+  install_template "php/opcache.ini.sh" "/etc/php/${PHPVER_NEW}/mods-available/opcache.ini"
+  run_app nc-limits
+  a2enconf "php${PHPVER_OLD}-fpm"
+  service "php${PHPVER_OLD}-fpm" start
+  service apache2 restart
+  echo "PHP upgrade has been successfully reverted"
+  set -e
+}
+
+trap php_restore INT TERM HUP ERR
+
+# Setup apt repository for php 8
+wget -O /etc/apt/trusted.gpg.d/php.gpg https://packages.sury.org/php/apt.gpg
+echo "deb https://packages.sury.org/php/ ${RELEASE%-security} main" > /etc/apt/sources.list.d/php.list
+apt-get update
+
+clear_opcache
+
+echo "Stopping apache and  php-fpm..."
+service "php${PHPVER_OLD}-fpm" stop
+service apache2 stop
+
+echo "Remove old PHP (${PHPVER_OLD})..."
+a2disconf "php${PHPVER_OLD}-fpm"
+
+apt-get remove --purge -y "${PHP_PACKAGES_OLD[@]}"
+
+echo "Install PHP ${PHPVER_NEW}..."
+install_with_shadow_workaround --no-install-recommends systemd
+apt-get install -y --no-install-recommends -t "$RELEASE" "${PHP_PACKAGES_NEW[@]}"
+
+set_ncpcfg "php_version" "${PHPVER_NEW}"
+install_template "php/opcache.ini.sh" "/etc/php/${PHPVER_NEW}/mods-available/opcache.ini"
+( set -e; export PHPVER="${PHPVER_NEW}"; run_app nc-limits )
+
+a2enconf "php${PHPVER_NEW}-fpm"
+
+echo "Starting apache and php-fpm..."
+service "php${PHPVER_NEW}-fpm" start
+service apache2 start
+ncc status

bin/ncp/BACKUPS/nc-export-ncp.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# Export NextCloudPi configuration
+# Export NextcloudPi configuration
 #
 #
 # Copyleft 2017 by Courtney Hicks

bin/ncp/BACKUPS/nc-import-ncp.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# Import NextCloudPi configuration
+# Import NextcloudPi configuration
 #
 #
 # Copyleft 2017 by Courtney Hicks

bin/ncp/BACKUPS/nc-snapshot-auto.sh

@@ -1,7 +1,7 @@
 #!/bin/bash
 
 #
-# NextCloudPi scheduled datadir BTRFS snapshots
+# NextcloudPi scheduled datadir BTRFS snapshots
 #
 # Copyleft 2017 by Ignacio Nunez Hernanz <nacho _a_t_ ownyourbits _d_o_t_ com>
 # GPL licensed (see end of file) * Use at your own risk!

bin/ncp/CONFIG/nc-database.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# Data dir configuration script for NextCloudPi
+# Data dir configuration script for NextcloudPi
 #
 # Copyleft 2017 by Ignacio Nunez Hernanz <nacho _a_t_ ownyourbits _d_o_t_ com>
 # GPL licensed (see end of file) * Use at your own risk!

bin/ncp/CONFIG/nc-datadir.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# Data dir configuration script for NextCloudPi
+# Data dir configuration script for NextcloudPi
 #
 # Copyleft 2017 by Ignacio Nunez Hernanz <nacho _a_t_ ownyourbits _d_o_t_ com>
 # GPL licensed (see end of file) * Use at your own risk!
@@ -103,7 +103,7 @@ configure()
   cd /var/www/nextcloud
   [[ "$BUILD_MODE" == 1 ]] || save_maintenance_mode
 
-  echo "moving data directory from ${SRCDIR} to ${BASEDIR}..."
+  echo "moving data directory from ${SRCDIR} to ${DATADIR}..."
 
   # use subvolumes, if BTRFS
   [[ "$(stat -fc%T "${BASEDIR}")" == "btrfs" ]] && ! is_docker && {
@@ -135,7 +135,7 @@ configure()
   create_tmp_upload_dir
   ncc config:system:set tempdirectory --value "$DATADIR/tmp" \
   || sed -i "s|'tempdirectory' =>.*|'tempdirectory' => '${DATADIR}/tmp',|" "${NCDIR?}"/config/config.php
-  sed -i "s|^;\?upload_tmp_dir =.*$|uploadtmp_dir = ${DATADIR}/tmp|"  /etc/php/"${PHPVER?}"/cli/php.ini
+  sed -i "s|^;\?upload_tmp_dir =.*$|upload_tmp_dir = ${DATADIR}/tmp|"  /etc/php/"${PHPVER?}"/cli/php.ini
   sed -i "s|^;\?upload_tmp_dir =.*$|upload_tmp_dir = ${DATADIR}/tmp|" /etc/php/"${PHPVER}"/fpm/php.ini
   sed -i "s|^;\?sys_temp_dir =.*$|sys_temp_dir = ${DATADIR}/tmp|"     /etc/php/"${PHPVER}"/fpm/php.ini
 

bin/ncp/CONFIG/nc-httpsonly.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# HTTPS rewrite configuration script for NextCloudPi
+# HTTPS rewrite configuration script for NextcloudPi
 #
 # Copyleft 2017 by Ignacio Nunez Hernanz <nacho _a_t_ ownyourbits _d_o_t_ com>
 # GPL licensed (see end of file) * Use at your own risk!

bin/ncp/CONFIG/nc-init.sh

@@ -53,10 +53,29 @@ EOF
   ## INITIALIZE NEXTCLOUD
 
   # make sure redis is running first
+  REDISPASS="$( grep "^requirepass" /etc/redis/redis.conf | cut -f2 -d' ' )"
   if ! pgrep -c redis-server &>/dev/null; then
     mkdir -p /var/run/redis
+    mkdir -p /var/log/
     chown redis /var/run/redis
-    sudo -u redis redis-server /etc/redis/redis.conf &
+    sudo -u redis redis-server /etc/redis/redis.conf > /var/log/redis.log 2>&1 &
+    redis_pid=$!
+    for i in {1..5}
+    do
+
+      if redis-cli -s //var/run/redis/redis.sock -a "$REDISPASS" ping | grep PONG
+      then
+        break
+      else
+        if [[ $i -ge 5 ]]
+        then
+          echo "FAILED TO START REDIS"
+          cat /var/log/redis.log /var/log/redis/redis-server.log
+          return 1
+        fi
+      fi
+      sleep 3
+    done
   fi
 
   while :; do
@@ -111,8 +130,8 @@ EOF
   # email
   ncc config:system:set mail_smtpmode     --value="sendmail"
   ncc config:system:set mail_smtpauthtype --value="LOGIN"
-  ncc config:system:set mail_from_address --value="admin"
-  ncc config:system:set mail_domain       --value="ownyourbits.com"
+  ncc config:system:set mail_from_address --value="noreply"
+  ncc config:system:set mail_domain       --value="nextcloudpi.com"
 
   # Fix NCP theme
   [[ -e /usr/local/etc/logo ]] && {
@@ -127,11 +146,11 @@ EOF
   }
 
   mysql nextcloud <<EOF
-replace into  oc_appconfig values ( 'theming', 'name'          , "NextCloudPi"             );
-replace into  oc_appconfig values ( 'theming', 'slogan'        , "keep your data close"    );
-replace into  oc_appconfig values ( 'theming', 'url'           , "https://ownyourbits.com" );
-replace into  oc_appconfig values ( 'theming', 'logoMime'      , "image/svg+xml"           );
-replace into  oc_appconfig values ( 'theming', 'backgroundMime', "image/png"               );
+replace into  ${DB_PREFIX}appconfig values ( 'theming', 'name'          , "NextCloudPi"             , 2, 0);
+replace into  ${DB_PREFIX}appconfig values ( 'theming', 'slogan'        , "keep your data close"    , 2, 0);
+replace into  ${DB_PREFIX}appconfig values ( 'theming', 'url'           , "https://nextcloudpi.com" , 2, 0);
+replace into  ${DB_PREFIX}appconfig values ( 'theming', 'logoMime'      , "image/svg+xml"           , 2, 0);
+replace into  ${DB_PREFIX}appconfig values ( 'theming', 'backgroundMime', "image/png"               , 2, 0);
 EOF
 
   # NCP app
@@ -152,11 +171,8 @@ EOF
   # we handle this ourselves
   ncc app:disable updatenotification
 
-  # News dropped support for 32-bit -> https://github.com/nextcloud/news/issues/1423
-  if ! [[ "$ARCH" =~ armv7 ]]; then
-    ncc app:install news
-    ncc app:enable  news
-  fi
+  # Not supported in Nextcloudpi without manual setup
+  ncc app:disable app_api
 
   # ncp-previewgenerator
   local ncver
@@ -195,6 +211,7 @@ EOF
   # TODO temporary workaround for https://github.com/nextcloud/server/pull/13358
   ncc -n db:convert-filecache-bigint
   ncc db:add-missing-indices
+  ncc maintenance:repair --include-expensive
 
   # Default trusted domain (only from ncp-config)
   test -f /usr/local/bin/nextcloud-domain.sh && {

bin/ncp/CONFIG/nc-limits.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# System limits configuration for NextCloudPi
+# System limits configuration for NextcloudPi
 #
 # Copyleft 2017 by Ignacio Nunez Hernanz <nacho _a_t_ ownyourbits _d_o_t_ com>
 # GPL licensed (see end of file) * Use at your own risk!
@@ -82,14 +82,14 @@ configure()
   [[ "$CONF_VALUE" == "$(cat "$CONF")" ]] || service mariadb restart
 
   # RESTART PHP
-  [[ "$require_fpm_restart" == "true" ]] && {
+  [[ "$require_fpm_restart" != "true" ]] || {
     bash -c "sleep 3; source /usr/local/etc/library.sh; clear_opcache; service php${PHPVER}-fpm restart" &>/dev/null &
   }
 
   # redis max memory
   local CONF=/etc/redis/redis.conf
   local CURRENT_REDIS_MEM="$( grep "^maxmemory" "$CONF" | awk '{ print $2 }' )"
-  [[ "$REDISMEM" != "$CURRENT_REDIS_MEM" ]] && {
+  [[ "$REDISMEM" == "$CURRENT_REDIS_MEM" ]] || {
     sed -i "s|^maxmemory .*|maxmemory $REDISMEM|" "$CONF"
     chown redis:redis "$CONF"
     service redis-server restart

bin/ncp/CONFIG/nc-nextcloud.sh

@@ -29,8 +29,8 @@ install()
   $APTINSTALL lbzip2 iputils-ping jq wget
   # NOTE: php-smbclient in sury but not in Debian sources, we'll use the binary version
   # https://docs.nextcloud.com/server/latest/admin_manual/configuration_files/external_storage/smb.html
-  $APTINSTALL -t $RELEASE smbclient exfat-fuse exfat-utils                      # for external storage
-  $APTINSTALL -t $RELEASE exfat-fuse exfat-utils                                # for external storage
+  $APTINSTALL -t $RELEASE smbclient exfat-fuse exfatprogs                      # for external storage
+  $APTINSTALL -t $RELEASE exfat-fuse exfatprogs                                # for external storage
   $APTINSTALL -t $RELEASE php${PHPVER}-exif                                     # for gallery
   $APTINSTALL -t $RELEASE php${PHPVER}-bcmath                                   # for LDAP
   $APTINSTALL -t $RELEASE php${PHPVER}-gmp                                      # for bookmarks
@@ -60,6 +60,10 @@ install()
   sed -i 's|# rename-command CONFIG ""|rename-command CONFIG ""|'  $REDIS_CONF
   sed -i "s|^port.*|port 0|"                                       $REDIS_CONF
   echo "maxmemory $REDIS_MEM" >> $REDIS_CONF
+  if [[ "$ARMBIAN_BUILD" == "yes" ]]
+  then
+    echo "ignore-warnings ARM64-COW-BUG" >> $REDIS_CONF
+  fi
   echo 'vm.overcommit_memory = 1' >> /etc/sysctl.conf
 
   if is_lxc; then
@@ -79,6 +83,25 @@ EOF
   update-rc.d redis-server enable
   clear_opcache
 
+  # NC service workers
+  cat > /etc/systemd/system/nextcloud-ai-worker@.service <<'EOF'
+[Unit]
+Description=Nextcloud AI worker %i
+After=network.target
+
+[Service]
+ExecStart=php occ background-job:worker -t 60 'OC\\TaskProcessing\\SynchronousBackgroundJob'
+Restart=always
+StartLimitInterval=60
+StartLimitBurst=10
+WorkingDirectory=/var/www/nextcloud
+User=www-data
+
+[Install]
+WantedBy=multi-user.target
+EOF
+
+
   # service to randomize passwords on first boot
   mkdir -p /usr/lib/systemd/system
   cat > /usr/lib/systemd/system/nc-provisioning.service <<'EOF'
@@ -194,11 +217,11 @@ EOF
 
 ## SET APACHE VHOST
   echo "Setting up Apache..."
-  install_template nextcloud.conf.sh /etc/apache2/sites-available/nextcloud.conf --allow-fallback || {
+  install_template nextcloud.conf.sh /etc/apache2/sites-available/001-nextcloud.conf --allow-fallback || {
       echo "ERROR: Parsing template failed. Nextcloud will not work."
       exit 1
   }
-  a2ensite nextcloud
+  a2ensite 001-nextcloud
 
   cat > /etc/apache2/sites-available/000-default.conf <<'EOF'
 <VirtualHost _default_:80>

bin/ncp/CONFIG/nc-passwd.sh

@@ -27,8 +27,9 @@ configure()
     sudo -u www-data php /var/www/nextcloud/cron.php
 
     a2dissite ncp-activation
-    a2ensite  ncp nextcloud
+    a2ensite  ncp 001-nextcloud
     apachectl -k graceful
+    sed -i -e 's/^NCP is not activated yet.*$//' /etc/issue
 
     # Trusted Domain (local/public IP), also configures notify_push
     bash /usr/local/bin/nextcloud-domain.sh

bin/ncp/CONFIG/nc-previews-auto.sh

@@ -23,8 +23,8 @@ configure()
   grep -qP "^\d+$" <<<"$RUNTIME" || { echo "Invalid RUNTIME value $RUNTIME"; return 1; }
   RUNTIME=$((RUNTIME*60))
 
-  echo "0  2  *  *  *  root  /usr/local/bin/nc-previews" >  /etc/cron.d/ncp-previews-auto
-  chmod 644 /etc/cron.d/ncp-previews-auto
+  echo "0  2  *  *  *  root  /usr/local/bin/nc-previews" >  /etc/cron.d/nc-previews-auto
+  chmod 644 /etc/cron.d/nc-previews-auto
 
   cat > /usr/local/bin/nc-previews <<EOF
 #!/bin/bash

bin/ncp/CONFIG/nc-trusted-domains.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# Manually add trusted domains in NextCloudPi
+# Manually add trusted domains in NextcloudPi
 #
 # Copyleft 2019 by Ignacio Nunez Hernanz <nacho _a_t_ ownyourbits _d_o_t_ com>
 # GPL licensed (see end of file) * Use at your own risk!

bin/ncp/CONFIG/ncp-community.sh

@@ -0,0 +1,7 @@
+#!/bin/bash
+
+# Configure various settings for community participation
+
+install() { :; }
+
+configure() { :; }

bin/ncp/NETWORKING/SSH.sh

@@ -11,6 +11,13 @@
 install() {
   apt-get update
   apt-get install -y --no-install-recommends openssh-server
+  if grep '^PermitRootLogin' /etc/ssh/sshd_config
+  then
+    sed -i -e 's/^PermitRootLogin.*$/PermitRootLogin prohibit-password/' /etc/ssh/sshd_config
+  else
+    echo 'PermitRootLogin prohibit-password' >> /etc/ssh/sshd_config
+  fi
+  systemctl reload ssh
  }
 
 is_active()

bin/ncp/NETWORKING/dnsmasq.sh

@@ -14,40 +14,41 @@ install()
   set -x
   apt-get update
   apt-get install --no-install-recommends -y dnsmasq
+  sleep 10
   rc=0
   service dnsmasq status > /dev/null 2>&1 || rc=$?
-  ! is_docker && [[ $rc -eq 3 ]] && ! [[ "$INIT_SYSTEM" =~ ^("chroot"|"unknown")$ ]] && {
+  if [[ $rc -eq 3 ]] && ! [[ "$INIT_SYSTEM" =~ ^("chroot"|"unknown")$ ]]
+  then
     echo "Applying workaround for dnsmasq bug (compare issue #1446)"
-    service systemd-resolved stop || true
-    service dnsmasq start
-    service dnsmasq status
-  }
+    mkdir -p /etc/systemd/resolved.conf.d
+    if systemctl status systemd-resolved
+    then
+  cat <<EOF > /etc/systemd/resolved.conf.d/nostublistener.conf
+[Resolve]
+DNSStubListener=no
+EOF
+      [[ "$INIT_SYSTEM" != "systemd" ]] || systemctl restart systemd-resolved
+    elif systemctl status resolvconf
+    then
+      systemctl stop resolvconf
+      systemctl start dnsmasq
+      systemctl status dnsmasq
+    else
+      echo "dnsmasq failed to start and no workaround could be found. This means, the installer failed."
+      false
+    fi
+#    service systemd-resolved stop || true
+    systemctl start dnsmasq
+    systemctl status dnsmasq
+  fi
 
   service dnsmasq stop
-  [[ "$INIT_SYSTEM" == "systemd" ]] && service systemd-resolved start || true
+  if [[ "$INIT_SYSTEM" == "systemd" ]] && systemctl list-unit-files resolvconf.service
+  then
+    systemctl start resolvconf
+  fi
   update-rc.d dnsmasq disable || rm /etc/systemd/system/multi-user.target.wants/dnsmasq.service
 
-  [[ "$DOCKERBUILD" == 1 ]] && {
-    cat > /etc/services-available.d/100dnsmasq <<EOF
-#!/bin/bash
-
-source /usr/local/etc/library.sh
-
-[[ "\$1" == "stop" ]] && {
-  echo "stopping dnsmasq..."
-  service dnsmasq stop
-  exit 0
-}
-
-persistent_cfg /etc/dnsmasq.conf
-
-echo "Starting dnsmasq..."
-service dnsmasq start
-
-exit 0
-EOF
-    chmod +x /etc/services-available.d/100dnsmasq
-  }
   return 0
 }
 

bin/ncp/NETWORKING/duckDNS.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# DuckDNS installation on Raspbian for NextCloudPi
+# DuckDNS installation on Raspbian for NextcloudPi
 #
 #
 # Copyleft 2017 by Courtney Hicks

bin/ncp/NETWORKING/letsencrypt.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# Let's encrypt certbot installation on NextCloudPi
+# Let's encrypt certbot installation on NextcloudPi
 #
 # Copyleft 2017 by Ignacio Nunez Hernanz <nacho _a_t_ ownyourbits _d_o_t_ com>
 # GPL licensed (see end of file) * Use at your own risk!
@@ -9,7 +9,7 @@
 
 
 ncdir=/var/www/nextcloud
-nc_vhostcfg=/etc/apache2/sites-available/nextcloud.conf
+nc_vhostcfg=/etc/apache2/sites-available/001-nextcloud.conf
 ncp_vhostcfg=/etc/apache2/sites-available/ncp.conf
 letsencrypt=/usr/bin/letsencrypt
 

bin/ncp/NETWORKING/nc-forward-ports.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# Use uPnP to forward router ports for NextCloudPi
+# Use uPnP to forward router ports for NextcloudPi
 #
 # Copyleft 2017 by Ignacio Nunez Hernanz <nacho _a_t_ ownyourbits _d_o_t_ com>
 # GPL licensed (see end of file) * Use at your own risk!

bin/ncp/NETWORKING/nc-trusted-proxies.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# Manually add trusted proxies in NextCloudPi
+# Manually add trusted proxies in NextcloudPi
 #
 # Copyleft 2019 by Pascal Haefliger <45995338+paschaef_a_t_users_d_o_t_noreply_d_o_tgithub_d_o_t_com>
 # GPL licensed (see end of file) * Use at your own risk!

bin/ncp/NETWORKING/no-ip.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# no-ip.org installation on NextCloudPi
+# no-ip.org installation on NextcloudPi
 #
 # Copyleft 2017 by Ignacio Nunez Hernanz <nacho _a_t_ ownyourbits _d_o_t_ com>
 # GPL licensed (see end of file) * Use at your own risk!
@@ -14,8 +14,8 @@ install()
   apt-get update
   apt-get install --no-install-recommends -y make gcc libc-dev
 
-  local TMPDIR="$( mktemp -d /tmp/noip.XXXXXX )"
-  cd "$TMPDIR"
+  local TEMPDIR="$( mktemp -d /tmp/noip.XXXXXX )"
+  cd "$TEMPDIR"
   wget -O- --content-disposition https://github.com/nachoparker/noip-DDNS/archive/master/latest.tar.gz \
   | tar -xz \
   || return 1
@@ -41,7 +41,7 @@ EOF
 
   chmod +x /etc/init.d/noip2
   cd -
-  rm -r "$TMPDIR"
+  rm -r "$TEMPDIR"
 
   update-rc.d noip2 defaults
   update-rc.d noip2 disable

bin/ncp/NETWORKING/spDYN.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# spDYN setup for NextCloudPi
+# spDYN setup for NextcloudPi
 #
 #
 # Copyleft 2017/2018 by Timm Goldenstein and Timo Stiefel

bin/ncp/SECURITY/fail2ban.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# Fail2ban for NextCloudPi
+# Fail2ban for NextcloudPi
 #
 # Copyleft 2017 by Ignacio Nunez Hernanz <nacho _a_t_ ownyourbits _d_o_t_ com>
 # GPL licensed (see end of file) * Use at your own risk!
@@ -114,7 +114,7 @@ protocol   = tcp
 chain      = INPUT
 action_    = %(banaction)s[name=%(__name__)s, port="%(port)s", protocol="%(protocol)s", chain="%(chain)s"]
 action_mwl = %(banaction)s[name=%(__name__)s, port="%(port)s", protocol="%(protocol)s", chain="%(chain)s"]
-           sendmail-whois-lines[name=%(__name__)s, dest=$EMAIL, sender=ncp-fail2ban@ownyourbits.com]
+           sendmail-whois-lines[name=%(__name__)s, dest=$EMAIL, sender=ncp-fail2ban@nextcloudpi.com]
 action = %($ACTION)s
 
 #

bin/ncp/SECURITY/nc-audit.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# Launch security audit reports for NextCloudPi
+# Launch security audit reports for NextcloudPi
 #
 # Copyleft 2017 by Ignacio Nunez Hernanz <nacho _a_t_ ownyourbits _d_o_t_ com>
 # GPL licensed (see end of file) * Use at your own risk!

bin/ncp/SECURITY/nc-encrypt.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# Data at rest encryption for NextCloudPi
+# Data at rest encryption for NextcloudPi
 #
 # Copyleft 2021 by Ignacio Nunez Hernanz <nacho _a_t_ ownyourbits _d_o_t_ com>
 # GPL licensed (see end of file) * Use at your own risk!
@@ -20,14 +20,14 @@ install()
 
 configure()
 {
-(
+
   set -e -o pipefail
   local datadir parentdir encdir tmpdir
   datadir="$(get_ncpcfg datadir)"
   [[ "${datadir?}" == "null" ]] && datadir=/var/www/nextcloud/data
   parentdir="$(dirname "${datadir}")"
   encdir="${parentdir?}/ncdata_enc"
-  tmpdir="$(mktemp -u -p "${parentdir}" -t nc-data-crypt.XXXXXX))"
+  tmpdir="$(mktemp -u -p "${parentdir}" -t nc-data-crypt.XXXXXX)"
 
   [[ "${ACTIVE?}" != "yes" ]] && {
     if ! is_active; then
@@ -55,12 +55,14 @@ configure()
     return
   fi
 
+  export PASSWORD
   # Just mount already encrypted data
   if [[ -f "${encdir?}"/gocryptfs.conf ]]; then
-    echo "${PASSWORD?}" | gocryptfs -allow_other -q "${encdir}" "${datadir}" 2>&1 | sed /^Switch/d
+    systemctl reset-failed ncp-encrypt 2>/dev/null ||:
+    systemd-run -u ncp-encrypt -E PASSWORD bash -c "gocryptfs -fg -allow_other -q '${encdir}' '${datadir}' <<<\"\${PASSWORD}\" 2>&1 | sed /^Switch/d |& tee /var/log/ncp-encrypt.log"
 
     # switch to the regular virtual hosts after we decrypt, so we can access NC and ncp-web
-    a2ensite ncp nextcloud
+    a2ensite ncp 001-nextcloud
     a2dissite ncp-activation
     apache2ctl -k graceful
 
@@ -70,12 +72,33 @@ configure()
   mkdir -p "${encdir?}"
   echo "${PASSWORD?}" | gocryptfs -init -q "${encdir}"
   save_maintenance_mode
+  cleanup() {
+    umount "${datadir}" ||:
+    [[ -f "${tmpdir}" ]] && {
+      rm -rf "${datadir?}" ||:
+      mv "${tmpdir}" "${datadir}"
+
+      chown -R www-data:www-data "${datadir}"
+    }
+  }
+
+  trap cleanup 1
   trap restore_maintenance_mode EXIT
 
   mv "${datadir?}" "${tmpdir?}"
 
   mkdir "${datadir}"
-  echo "${PASSWORD}" | gocryptfs -allow_other -q "${encdir}" "${datadir}" 2>&1 | sed /^Switch/d
+  systemctl reset-failed ncp-encrypt 2>/dev/null ||:
+  systemd-run -u ncp-encrypt -E PASSWORD bash -c "gocryptfs -fg -allow_other -q '${encdir}' '${datadir}' <<<\"\${PASSWORD}\" 2>&1 | sed /^Switch/d |& tee /var/log/ncp-encrypt.log"
+
+  maxtries=5
+  while [[ "$(systemctl is-active ncp-encrypt)" != "active" ]] || ! mount | grep -1 "${datadir}"
+  do
+    echo "Wating for encryption process to start... (${maxtries})"
+    sleep 3
+    maxtries=$((maxtries - 1))
+    [[ $maxtries -gt 0 ]] || return 1
+  done
 
   echo "Encrypting data..."
   mv "${tmpdir}"/* "${tmpdir}"/.[!.]* "${datadir}"
@@ -85,7 +108,7 @@ configure()
   set_ncpcfg datadir "${datadir}"
 
   echo "Data is now encrypted"
-)
+
 }
 
 # License

bin/ncp/SYSTEM/metrics.sh

@@ -196,8 +196,8 @@ configure() {
   if [[ "$ACTIVE" != yes ]]
   then
 
-    install_template nextcloud.conf.sh /etc/apache2/sites-available/nextcloud.conf || {
-      install_template nextcloud.conf.sh /etc/apache2/sites-available/nextcloud.conf --allow-fallback
+    install_template nextcloud.conf.sh /etc/apache2/sites-available/001-nextcloud.conf || {
+      install_template nextcloud.conf.sh /etc/apache2/sites-available/001-nextcloud.conf --allow-fallback
       echo -e "ERROR while generating nextcloud.conf! Exiting..."
       return 1
     }
@@ -248,8 +248,8 @@ configure() {
     echo "done."
 
 
-    install_template nextcloud.conf.sh /etc/apache2/sites-available/nextcloud.conf || {
-      install_template nextcloud.conf.sh /etc/apache2/sites-available/nextcloud.conf --allow-fallback
+    install_template nextcloud.conf.sh /etc/apache2/sites-available/001-nextcloud.conf || {
+      install_template nextcloud.conf.sh /etc/apache2/sites-available/001-nextcloud.conf --allow-fallback
       echo -e "ERROR while generating nextcloud.conf! Exiting..."
       return 1
     }

bin/ncp/SYSTEM/nc-automount.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# Automount configuration for NextCloudPi
+# Automount configuration for NextcloudPi
 #
 # Copyleft 2017 by Ignacio Nunez Hernanz <nacho _a_t_ ownyourbits _d_o_t_ com>
 # GPL licensed (see end of file) * Use at your own risk!

bin/ncp/SYSTEM/nc-ramlogs.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# Data dir configuration script for NextCloudPi
+# Data dir configuration script for NextcloudPi
 #
 # Copyleft 2017 by Ignacio Nunez Hernanz <nacho _a_t_ ownyourbits _d_o_t_ com>
 # GPL licensed (see end of file) * Use at your own risk!

bin/ncp/SYSTEM/nc-swapfile.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# Data dir configuration script for NextCloudPi
+# Data dir configuration script for NextcloudPi
 #
 # Copyleft 2017 by Ignacio Nunez Hernanz <nacho _a_t_ ownyourbits _d_o_t_ com>
 # GPL licensed (see end of file) * Use at your own risk!

bin/ncp/SYSTEM/nc-zram.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# NextCloudPi ZRAM settings
+# NextcloudPi ZRAM settings
 #
 # Copyleft 2017 by Ignacio Nunez Hernanz <nacho _a_t_ ownyourbits _d_o_t_ com>
 # GPL licensed (see end of file) * Use at your own risk!

bin/ncp/TOOLS/nc-format-USB.sh

@@ -51,7 +51,7 @@ configure()
   }
 
   DATADIR="$(get_nc_config_value datadirectory || true)"
-  if [[ $( stat -fc%d / ) != $( stat -fc%d "$DATADIR" ) ]] || [[ -z "$DATADIR" ]] && [[ "$ALLOW_DATA_DIR_REMOVAL" != "yes" ]]
+  if [[ $( stat -fc%d / ) != $( stat -fc%d "$DATADIR" ) ]] || [[ -z "$DATADIR" ]] && [[ "$ALLOW_DATADIR_REMOVAL" != "yes" ]]
   then
     echo "ERROR: Data directory is on USB drive (or can't be determined) and removal of data directory was not explicitly allowed." \
       "Please move the data directory to SD before formatting the USB drive." \

bin/ncp/TOOLS/nc-previews.sh

@@ -24,7 +24,7 @@ configure()
     }
 
     rm -r "$datadir"/appdata_*/preview/* &>/dev/null
-    mysql nextcloud <<<"delete from oc_filecache where path like \"appdata_%/preview/%\""
+    mysql nextcloud <<<"delete from ${DB_PREFIX?}filecache where path like \"appdata_%/preview/%\""
     ncc files:scan-app-data -n
   }
 

bin/ncp/UPDATES/nc-autoupdate-nc.sh

@@ -23,11 +23,11 @@ configure()
 source /usr/local/etc/library.sh
 
 echo -e "[ncp-update-nc]"                              >> /var/log/ncp.log
-/usr/local/bin/ncp-update-nc "$NCLATESTVER" 2>&1 | tee -a /var/log/ncp.log
+/usr/local/bin/ncp-update-nc "latest" 2>&1 | tee -a /var/log/ncp.log
 
 if [[ \${PIPESTATUS[0]} -eq 0 ]]; then
 
-  VER="\$( /usr/local/bin/ncc status | grep "version:" | awk '{ print \$3 }' )"
+  VER="\$(nc_version)"
 
   notify_admin "NextCloudPi" "Nextcloud was updated to \$VER"
 fi

bin/ncp/UPDATES/nc-autoupdate-ncp.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# Automatically apply NextCloudPi updates
+# Automatically apply NextcloudPi updates
 #
 # Copyleft 2017 by Ignacio Nunez Hernanz <nacho _a_t_ ownyourbits _d_o_t_ com>
 # GPL licensed (see end of file) * Use at your own risk!

bin/ncp/UPDATES/nc-update-nextcloud.sh

@@ -12,12 +12,6 @@ LATEST="$NCLATESTVER"
 
 configure()
 {
-  [[ "$VERSION" == "0" ]] && VERSION="$LATEST"
-  if ! is_docker && ! is_more_recent_than "24.0.0" "${VERSION}" && is_more_recent_than "8.1.0" "${PHPVER}.0" && [[ " ${BASH_SOURCE[*]} " =~ .*" /home/www/ncp-launcher.sh ".* ]]
-    then
-      echo "We need to upgrade PHP. This process cannot be performed from the web UI. Please use 'ncp-config' from the terminal (via SSH or direct access) to update Nextcloud instead. Future updates can again be run from the web UI"
-      exit 1
-    fi
   bash /usr/local/bin/ncp-update-nc "$VERSION"
 }
 

bin/ncp/UPDATES/nc-update.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# Data dir configuration script for NextCloudPi
+# Data dir configuration script for NextcloudPi
 #
 # Copyleft 2017 by Ignacio Nunez Hernanz <nacho _a_t_ ownyourbits _d_o_t_ com>
 # GPL licensed (see end of file) * Use at your own risk!

bin/ncp/UPDATES/unattended-upgrades.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# Unattended upgrades installation on NextCloudPi
+# Unattended upgrades installation on NextcloudPi
 #
 # Copyleft 2017 by Ignacio Nunez Hernanz <nacho _a_t_ ownyourbits _d_o_t_ com>
 # GPL licensed (see end of file) * Use at your own risk!

bin/nextcloud-domain.sh

@@ -11,9 +11,12 @@ done
 # wicd service finishes before completing DHCP
 while :; do
   local_ip="$(get_ip)"
-  pub_ip="$(curl -m4 icanhazip.com 2>/dev/null)"
 
-  [[ "$pub_ip"   != "" ]] && ncc config:system:set trusted_domains 11 --value="$pub_ip"
+  pub_ipv4="$(curl -4 -m4 icanhazip.com 2>/dev/null)"
+  pub_ipv6="$(curl -6 -m4 icanhazip.com 2>/dev/null)"
+  [[ -z "$pub_ipv4" ]] || ncc config:system:set trusted_domains 11 --value="$pub_ipv4"
+  [[ -z "$pub_ipv6" ]] || ncc config:system:set trusted_domains 12 --value="[$pub_ipv6]"
+
   [[ "$local_ip" != "" ]] && break
 
   sleep 3

build/armbian/Vagrantfile

@@ -0,0 +1,43 @@
+# -*- mode: ruby -*-
+# vi: set ft=ruby :
+
+#
+# Vagrantfile for the NCP Debian VM
+#
+# Instructions: vagrant up; vagrant ssh
+#
+# Notes: User/Pass is ubnt/ubnt.
+# $HOME is accessible as /external. CWD is accessible as /cwd
+#
+
+Vagrant.configure("2") do |config|
+
+  vmname = "NCP Ubuntu VM"
+  config.vm.box = "generic/ubuntu2204"
+  config.vm.box_check_update = true
+  config.vm.hostname = "ncp-vm"
+  config.vm.define "ncp-vm"
+  config.vm.provider :libvirt do |libvirt|
+    libvirt.default_prefix = ""
+    libvirt.cpus = 6
+    libvirt.memory = 4096
+  end
+
+  config.vm.synced_folder '.', '/vagrant', disabled: true
+
+  $script = <<-SHELL
+    sudo su
+    set -ex
+    DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends git ca-certificates
+    cd /tmp/nextcloudpi
+    ./build/build-SD-armbian.sh "${BOARD_ID?}" "${BOARD_NAME?}"
+    mv /tmp/nextcloudpi/armbian/output/*.img /tmp/ncp-out/
+  SHELL
+
+  # Provision the VM
+  config.vm.provision "shell", inline: $script, env: {"BOARD_ID" => ENV['BOARD_ID'], "BOARD_NAME" => ENV['BOARD_NAME']}
+  config.vm.synced_folder ".", "/vagrant", disabled: true
+  config.vm.synced_folder "../..", "/tmp/nextcloudpi", type: "rsync", rsync__exclude: ["/armbian/", ".venv/", "/cache/", "/raspbian_root/", "/output/", "/tmp/", "/.idea/"]
+  config.vm.synced_folder "../../output", "/tmp/ncp-out", type: "sshfs"
+
+end

build/armbian/armbian.sh

@@ -12,11 +12,15 @@
 
 set -e
 
+echo 'Running user script...'
+
 ARMBIAN_RELEASE=$1
 LINUXFAMILY=$2
 BOARD=$3
 BUILD_DESKTOP=$4
 
+echo "include home dir? ${INCLUDE_HOME_DIR:-no}"
+
 cd /tmp/overlay
 NCPCFG=etc/ncp.cfg
 source etc/library.sh # sets RELEASE
@@ -29,12 +33,24 @@ echo -e "\nInstalling NextCloudPi"
 
 hostname -F /etc/hostname # fix 'sudo resolve host' errors
 
-CODE_DIR="$(pwd)" bash install.sh
+ARMBIAN_BUILD=yes CODE_DIR="$(pwd)" DBG=x bash install.sh || {
+  echo "SOMETHING WENT WRONG, EXITING..."
+  exit 1
+}
+sed -i 's/^ignore-warnings ARM64-COW-BUG//' /etc/redis/redis.conf
+
+echo -e "\nPostinstall..."
 run_app_unsafe post-inst.sh
 
 # disable SSH by default, it can be enabled through ncp-web
 systemctl disable ssh
 
+# disable armbian autologin
+rm -f /etc/systemd/system/getty@.service.d/override.conf
+rm -f /etc/systemd/system/serial-getty@.service.d/override.conf
+rm -f /root/.not_logged_in_yet
+sed -i 's|^root::|root:x:|' /etc/passwd
+
 basename "$IMG" | tee /usr/local/etc/ncp-baseimage
 
 cd -

build/armbian/armbian_version

@@ -0,0 +1 @@
+v25.08

build/build-LXC.sh

@@ -14,7 +14,7 @@ source build/buildlib.sh
 echo -e "\e[1m\n[ Build NCP LXC ]\e[0m"
 
 #CLEAN=0                    # Pass this envvar to skip cleaning download cache
-IMG="NextCloudPi_LXC_$( date  "+%m-%d-%y" ).img"
+IMG="NextcloudPi_LXC_$( date  "+%m-%d-%y" ).img"
 IMG=tmp/"$IMG"
 
 TAR=output/"$( basename "$IMG" .img ).tar.bz2"

build/build-LXD.sh

@@ -14,7 +14,7 @@ source build/buildlib.sh
 echo -e "\e[1m\n[ Build NCP LXD ]\e[0m"
 
 #CLEAN=0                    # Pass this envvar to skip cleaning download cache
-IMG="NextCloudPi_LXD_$( date  "+%m-%d-%y" ).img"
+IMG="NextcloudPi_LXD_$( date  "+%m-%d-%y" ).img"
 IMG=tmp/"$IMG"
 
 TAR=output/"$( basename "$IMG" .img ).tar.bz2"
@@ -31,15 +31,22 @@ prepare_dirs                   # tmp cache output
 
 ## BUILD NCP
 
-lxc delete -f ncp 2>/dev/null || true
-LXC_CREATE=(lxc init -p default)
+debian_version="$(. etc/library.sh > /dev/null 2>&1; echo "${RELEASE%%-security}")"
+
+LXC_CMD=(lxc)
+[[ "$USE_INCUS" == "yes" ]] && LXC_CMD=(incus)
+
+"${LXC_CMD[@]}" info || LXC_CMD=(sudo "${LXC_CMD[0]}")
+
+"${LXC_CMD[@]}" delete -f ncp 2>/dev/null || true
+LXC_CREATE=("${LXC_CMD[@]}" init -p default)
 [[ -n "$LXD_EXTRA_PROFILE" ]] && LXC_CREATE+=(-p "$LXD_EXTRA_PROFILE")
 if [[ -n "$LXD_ARCH" ]] && [[ "$LXD_ARCH" != "x86" ]]
 then
   echo "Building for architecture: $LXD_ARCH"
-  LXC_CREATE+=("images:debian/bullseye/$LXD_ARCH")
+  LXC_CREATE+=("images:debian/${debian_version}/$LXD_ARCH")
 else
-  LXC_CREATE+=('images:debian/bullseye')
+  LXC_CREATE+=("images:debian/${debian_version}")
 fi
 LXC_CREATE+=(ncp)
 "${LXC_CREATE[@]}"
@@ -56,19 +63,26 @@ LXC_CREATE+=(ncp)
 #  fi
 #fi
 
-systemd-run --user --scope -p "Delegate=yes" lxc start ncp -q || \
-sudo systemd-run --scope -p "Delegate=yes" lxc start ncp -q
-lxc config device add ncp buildcode disk source="$(pwd)" path=/build
-lxc exec ncp -- bash -c 'while [ "$(systemctl is-system-running 2>/dev/null)" != "running" ] && [ "$(systemctl is-system-running 2>/dev/null)" != "degraded" ]; do :; done'
-lxc exec ncp -- bash -c 'CODE_DIR=/build DBG=x bash /build/install.sh'
-lxc exec ncp -- bash -c 'source /build/etc/library.sh; run_app_unsafe /build/post-inst.sh'
-lxc exec ncp -- bash -c "echo '$(basename "$IMG")' > /usr/local/etc/ncp-baseimage"
-lxc stop ncp
-lxc config device remove ncp buildcode
-lxc publish -q ncp -f --alias ncp/"${version}"
+set -x
+EXEC_ARGS=()
+[[ -z "$BRANCH" ]] || EXEC_ARGS+=(--env "BRANCH=${BRANCH}")
+systemd-run --user --scope -p "Delegate=yes" "${LXC_CMD[@]}" start ncp -q || \
+sudo systemd-run --scope -p "Delegate=yes" "${LXC_CMD[@]}" start ncp -q || {
+  rc=$?
+  "${LXC_CMD[@]}" info --show-log ncp
+  exit $rc
+}
+"${LXC_CMD[@]}" config device add ncp buildcode disk source="$(pwd)" path=/build
+"${LXC_CMD[@]}" exec ncp "${EXEC_ARGS[@]}" -- bash -c 'while [ "$(systemctl is-system-running 2>/dev/null)" != "running" ] && [ "$(systemctl is-system-running 2>/dev/null)" != "degraded" ]; do :; done'
+"${LXC_CMD[@]}" exec ncp "${EXEC_ARGS[@]}" -- bash -c 'CODE_DIR=/build DBG=x bash /build/install.sh'
+"${LXC_CMD[@]}" exec ncp "${EXEC_ARGS[@]}" -- bash -c 'source /build/etc/library.sh; run_app_unsafe /build/post-inst.sh'
+"${LXC_CMD[@]}" exec ncp "${EXEC_ARGS[@]}" -- bash -c "echo '$(basename "$IMG")' > /usr/local/etc/ncp-baseimage"
+"${LXC_CMD[@]}" stop ncp
+"${LXC_CMD[@]}" config device remove ncp buildcode
+"${LXC_CMD[@]}" publish -q ncp -f --alias ncp/"${version}"
 
 ## pack
-[[ " $* " =~ .*" --pack ".* ]] && lxc image export -q ncp/"${version}" "$TAR"
+[[ " $* " =~ .*" --pack ".* ]] && "${LXC_CMD[@]}" image export -q ncp/"${version}" "$TAR"
 
 exit 0
 

build/build-SD-armbian-vagrant.sh

@@ -0,0 +1,30 @@
+#!/bin/bash
+
+# Batch creation of NextCloudPi Armbian based images
+#
+# Copyleft 2023 by Tobias Knöppler
+# GPL licensed (see end of file) * Use at your own risk!
+#
+# Usage: sudo ./build-SD-armbian-vagrant.sh <board_code> [<board_name>]
+#
+
+[[ "$UID" == 0 ]] || {
+  echo "This script needs to be run as root. Try sudo"
+  exit 1
+}
+
+set -ex
+export BOARD_ID="${1?}"
+export BOARD_NAME="${2:-$1}"
+vagrant plugin list | grep vagrant-libvirt || vagrant plugin install vagrant-libvirt
+vagrant plugin list | grep vagrant-sshfs || vagrant plugin install vagrant-sshfs
+export VAGRANT_DEFAULT_PROVIDER=libvirt
+
+vagrant box list | grep generic/ubuntu2204 || vagrant box add --provider libvirt generic/ubuntu2204
+
+cd "$(dirname "$0")/armbian"
+mkdir -p "../../output"
+trap 'echo "Cleaning up vagrant..."; vagrant halt; vagrant destroy -f' EXIT
+BOARD_ID="$BOARD_ID" BOARD_NAME="$BOARD_NAME" vagrant up --provider=libvirt
+
+exit 0

build/build-SD-armbian.sh

@@ -18,7 +18,7 @@ NCPCFG=etc/ncp.cfg
 
 echo -e "\e[1m\n[ Build NCP ${BNAME} ]\e[0m"
 
-IMG="${IMG:-NextCloudPi_${BNAME}_$( date  "+%m-%d-%y" ).img}"
+IMG="${IMG:-NextcloudPi_${BNAME}_$( date  "+%m-%d-%y" ).img}"
 IMG=tmp/"$IMG"
 TAR=output/"$( basename "$IMG" .img ).tar.bz2"
 
@@ -30,8 +30,13 @@ source etc/library.sh # sets RELEASE
 prepare_dirs                   # tmp cache output
 
 # get latest armbian
-[[ -d armbian ]] || git clone https://github.com/armbian/build armbian
-( cd armbian && git pull --ff-only --tags && git checkout v23.02 )
+[[ -d armbian ]] || {
+  git clone --depth 1 --branch "$(cat "$(dirname "${0}")/armbian/armbian_version")" https://github.com/armbian/build armbian
+}
+#( cd armbian && git pull --ff-only --tags && git checkout v23.02 )
+#sed -i -e '/export rootfs_size=/s/du -sm/du --apparent-size -sm/' armbian/lib/functions/image/partitioning.sh
+#sed -i -e '/export rootfs_size_mib=/s/du -sm/du --apparent-size -sm/' armbian/lib/functions/main/rootfs-image.sh
+#sed -i -e 's/du /du --apparent-size /' armbian/lib/functions/image/rootfs-to-image.sh
 
 # add NCP modifications
 mkdir -p armbian/userpatches armbian/userpatches/overlay
@@ -48,11 +53,11 @@ cat > "$CONF" <<EOF
 BOARD="$BOARD"
 BRANCH=current
 RELEASE=${RELEASE%%-security}
-KERNEL_ONLY=no
 KERNEL_CONFIGURE=prebuilt
 BUILD_DESKTOP=no
 BUILD_MINIMAL=yes
 USE_CCACHE=yes
+INCLUDE_HOME_DIR=yes
 EOF
 [[ "$CLEAN" == "0" ]] && {
   cat >> "$CONF" <<EOF
@@ -68,10 +73,11 @@ EXTRA_CONF=build/armbian/"config-$BOARD".conf
 # build
 rm -rf armbian/output/images
 mkdir -p armbian/userpatches
-sed -e '/docker.*run/s/-it//' armbian/config/templates/config-docker.conf > armbian/userpatches/config-docker.conf
-docker pull "ghcr.io/armbian/build:$(cut -d"." -f1-2 < armbian/VERSION)-$(dpkg --print-architecture)" \
- || docker pull "ghcr.io/armbian/build:latest-$(dpkg --print-architecture)"
-sudo armbian/compile.sh docker ncp
+#sed -e '/docker.*run/s/-it//' armbian/config/templates/config-docker.conf > armbian/userpatches/config-docker.conf
+#docker pull "ghcr.io/armbian/build:$(cut -d"." -f1-2 < armbian/VERSION)-$(dpkg --print-architecture)" \
+# || docker pull "ghcr.io/armbian/build:latest-$(dpkg --print-architecture)"
+export INCLUDE_HOME_DIR=yes
+armbian/compile.sh ncp
 rm "$CONF"
 
 # pack image

build/build-SD-berryboot.sh

@@ -14,7 +14,7 @@ source build/buildlib.sh
 echo -e "\e[1m\n[ Build NCP Berryboot ]\e[0m"
 
 SRC="$1"
-IMG="NextCloudPi_RPi_Berryboot_$( date  "+%m-%d-%y" ).img"
+IMG="NextcloudPi_RPi_Berryboot_$( date  "+%m-%d-%y" ).img"
 TAR=output/"$( basename "$IMG" .img ).tar.bz2"
 
 test -f "$TAR" && { echo "$TAR already exists. Skipping... "; exit 0; }

build/build-SD-rpi.sh

@@ -16,7 +16,7 @@ echo -e "\e[1m\n[ Build NCP Raspberry Pi ]\e[0m"
 URL="https://downloads.raspberrypi.org/raspios_lite_arm64/images/raspios_lite_arm64-2022-09-26/2022-09-22-raspios-bullseye-arm64-lite.img.xz"
 SIZE=4G                     # Raspbian image size
 #CLEAN=0                    # Pass this envvar to skip cleaning download cache
-IMG="${IMG:-NextCloudPi_RPi_$( date  "+%m-%d-%y" ).img}"
+IMG="${IMG:-NextcloudPi_RPi_$( date  "+%m-%d-%y" ).img}"
 TAR=output/"$( basename "$IMG" .img ).tar.bz2"
 
 ##############################################################################

build/build-VM.sh

@@ -16,7 +16,7 @@ echo -e "\e[1m\n[ Build NCP VM ]\e[0m"
 IP=${1:-192.168.0.145}      # For QEMU automated testing (optional)
 SIZE=3G                     # Raspbian image size
 #CLEAN=0                    # Pass this envvar to skip cleaning download cache
-IMG="${IMG:-NextCloudPi_VM_$( date  "+%m-%d-%y" ).img}"
+IMG="${IMG:-NextcloudPi_VM_$( date  "+%m-%d-%y" ).img}"
 IMG=tmp/"$IMG"
 VM="/var/lib/libvirt/images/ncp-vm.img"
 

build/buildlib.sh

@@ -335,43 +335,6 @@ function generate_changelog()
     sed 's|* \[tag: |\n[|' > changelog.md
 }
 
-function upload_ftp()
-{
-  local IMGNAME="$1"
-  echo -e "\n\e[1m[ Upload FTP ]\e[0m"
-  echo "* $IMGNAME..."
-  [[ -f torrent/"$IMGNAME"/"$IMGNAME".tar.bz2 ]] || { echo "No image file found, abort"; return 1; }
-  [[ "$FTPPASS" == "" ]] && { echo "No FTPPASS variable found, skip upload"; return 0; }
-
-  cd torrent
-
-  ftp -np ftp.ownyourbits.com <<EOF
-user root@ownyourbits.com $FTPPASS
-mkdir testing
-mkdir testing/$IMGNAME
-cd testing/$IMGNAME
-binary
-rm  $IMGNAME.torrent
-put $IMGNAME.torrent
-bye
-EOF
-  cd -
-  cd torrent/$IMGNAME
-
-  ftp -np ftp.ownyourbits.com <<EOF
-user root@ownyourbits.com $FTPPASS
-cd testing/$IMGNAME
-binary
-rm  $IMGNAME.tar.bz2
-put $IMGNAME.tar.bz2
-rm  md5sum
-put md5sum
-bye
-EOF
-  ret=$?
-  cd -
-  return $ret
-}
 
 upload_images()
 {

changelog.md

@@ -1,5 +1,172 @@
 # NextcloudPi Changelog
 
+
+## [v1.56.0](https://github.com/nextcloud/nextcloudpi/tree/v1.56.0) (2025-11-19) Nextcloud 32 and fixes
+
+### Changes
+
+- Support for Nextcloud 32
+- Add service workers for Nextcloud's AI functionality
+
+### Fixes
+
+- Fix broken IPv6 addresses added to trusted IPs ([#2060](https://github.com/nextcloud/nextcloudpi/issues/2060), thanks @lukaszgoworko)
+- Fix unnecessary reinstallation of notify_push app during updates ([#2069](https://github.com/nextcloud/nextcloudpi/issues/2069), thanks @lovenemesis)
+
+## [v1.55.4](https://github.com/nextcloud/nextcloudpi/tree/v1.55.3) (2025-03-24) Nextcloud 31 and fixes
+
+### Changes
+
+- Support for Nextcloud 31
+
+### Fixes
+
+- Fix wrongly configured temporary directory for uploads (fixes [#1750](https://github.com/nextcloud/nextcloudpi/issues/1750), [#1857](https://github.com/nextcloud/nextcloudpi/issues/1857)), thanks [@joshtrichards](https://github.com/joshtrichards)
+- Fix regression breaking php upgrades (see [#2026](https://github.com/nextcloud/nextcloudpi/pull/2026)), thanks [@twojstaryzdomu](https://github.com/twojstaryzdomu)
+
+## [v1.55.3](https://github.com/nextcloud/nextcloudpi/tree/v1.55.3) (2024-11-08) Nextcloud 30 and fixes
+
+### Changes
+
+- Support for Nextcloud 30 ([#1957](https://github.com/nextcloud/nextcloudpi/issues/1957))
+
+### Fixes
+
+- Fix resetting failed state of previous ncp-update-nc job ([#1992](https://github.com/nextcloud/nextcloudpi/issues/1992))
+- Fix installer on systems using resolvconf as DNS manager ([#1926](](https://github.com/nextcloud/nextcloudpi/issues/1926))
+- Fix typo that prevents ncp-previews from being disabled (Thanks @m-breitbach)
+
+## [v1.55.2](https://github.com/nextcloud/nextcloudpi/tree/v1.55.2) (2024-09-24) Hotfix release
+
+### Fixes
+
+- Increase the maximum package size for mysqldump to 256M (fixes [#1979](https://github.com/nextcloud/nextcloudpi/issues/1979))
+- Fix repeated erroneous update success messages for auto updates (fixes [#1979](https://github.com/nextcloud/nextcloudpi/issues/1979), [#1981](https://github.com/nextcloud/nextcloudpi/issues/1981))
+
+## [v1.55.1](https://github.com/nextcloud/nextcloudpi/tree/v1.55.1) (2024-09-08) Hotfix release
+
+### Fixes
+
+- Prevent attempted downgrade in ncp-update-nc
+
+## [v1.55.0](https://github.com/nextcloud/nextcloudpi/tree/v1.55.0) (2024-09-07) Support for PHP 8.3 and Nextcloud 29
+
+### Changes
+
+- Support for Nextcloud 29
+- Support for PHP 8.3
+- Run `maintenance:repair --include-expensive` after NC update if necessary
+- Add logrotate for redis, apache and mysql logs
+- Add support for the OrangePi 5 and OrangePi 5 Plus boards
+
+### Fixes
+
+- Fix bug in nc-format-USB.sh (thanks @m-breitbach)
+- Nextcloud updater fixes 
+  - Fix constant retries of the auto update if on a version that's later than the last officially supported one [#1965](https://github.com/nextcloud/nextcloudpi/issues/1965)
+  - Fix incorrect detection of os constraints for NC updates beyond v28 [#1960](https://github.com/nextcloud/nextcloudpi/issues/1960)
+  - Fix user requested version being ignored if a later version was available
+- Improve stability of nc-encrypt by daemonizing the encryption process [#1874](https://github.com/nextcloud/nextcloudpi/issues/1874)
+
+### Development related
+
+- Perform nc update tests for both lxd as well as vms during CI/CD
+
+## [v1.54.3](https://github.com/nextcloud/nextcloudpi/tree/v1.54.3) (2024-08-11) Improved Nextcloud (auto) updates
+
+### Fixes
+
+- Prevent dependency conflict during distribution upgrade to debian 12
+
+### Changes
+
+- Nextcloud updates now automatically determine the next valid version to upgrade to. Auto updates always update to the latest minor version.
+
+## [v1.54.2](https://github.com/nextcloud/nextcloudpi/tree/v1.54.2) (2024-05-08) Fixes for ncp-dist-upgrade (bookworm)
+
+### Fixes
+
+- Make sure zstd is installed [#1921](https://github.com/nextcloud/nextcloudpi/issues/1921)
+- Dynamically determine DB prefix [#1818](https://github.com/nextcloud/nextcloudpi/issues/1818)
+
+## [v1.54.1](https://github.com/nextcloud/nextcloudpi/tree/v1.54.1) (2024-05-08) Fixes for ncp-dist-upgrade (bookworm)
+
+### Fixes
+
+- Reenable erroneously disabled apt package sources
+- Remove login message about available dist-upgrade if already on Debian 12
+
+## [v1.54.0](https://github.com/nextcloud/nextcloudpi/tree/v1.54.0) (2024-05-01) Support Debian 12 and Nextcloud 28.0.5
+
+### Changes
+
+- Add support for Nextcloud 28.0.5
+- Add ncp-dist-upgrade script to upgrade to Debian 12 (bookworm)
+- Change mechanism to prevent root login by default
+- Add support for Raspberry Pi 5
+- Use Armbian instead of Raspbian as base for Raspberry Pi images
+
+### Development related
+
+- (CICD) Add integration tests for dist-upgrade
+- Add support for building with incus
+
+## [v1.53.3](https://github.com/nextcloud/nextcloudpi/tree/v1.53.3) (2024-04-24) Support Nextcloud 28.0.4
+
+### Changes
+
+- Add support for Nextcloud 28.0.4
+
+
+## [v1.53.2](https://github.com/nextcloud/nextcloudpi/tree/v1.53.2) (2024-03-28) Support Nextcloud 28
+
+### Changes
+
+- Add support for Nextcloud 28.0.3
+
+
+## [v1.53.1](https://github.com/nextcloud/nextcloudpi/tree/v1.53.1) (2024-02-01) Support Nextcloud 27.1.6
+
+### Changes
+
+- Add support for Nextcloud 27.1.6
+
+### Development related
+
+- minor fixes to the build process
+
+## [v1.53.0](https://github.com/nextcloud/nextcloudpi/tree/v1.53.0) (2023-11-15) Staged Rollouts and NC 27.1.3
+
+### Changes
+
+- Add support for Nextcloud 27.1.3
+- Add support for staged rollouts
+- Add admin notification and community participation options
+- Add NCP section in Nextcloud settings 
+
+### Development related
+
+- Use latest armbian build stack (armbian next)
+- Support latest Selenium version in integration tests
+
+## [v1.52.4](https://github.com/nextcloud/nextcloudpi/tree/v1.52.4) (2023-07-31) Fix error in NCP updater
+
+### Fixes
+
+- Fix missing command in NCP updater
+
+## [v1.52.3](https://github.com/nextcloud/nextcloudpi/tree/v1.52.3) (2023-07-26) Add support for Nextcloud 27.0.1
+
+### Changes
+
+- Add support for Nextcloud 27.0.1
+
+## [v1.52.2](https://github.com/nextcloud/nextcloudpi/tree/v1.52.2) (2023-07-19) Hotfix: Fix crash in ncp-restore
+
+### Fixes
+
+- Fix crash in `ncp-restore`
+
 ## [v1.52.1](https://github.com/nextcloud/nextcloudpi/tree/v1.52.1) (2023-07-17) Add support for Nextcloud 26.0.3
 
 ### Changes

etc/library.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# NextCloudPi function library
+# NextcloudPi function library
 #
 # Copyleft 2017 by Ignacio Nunez Hernanz <nacho _a_t_ ownyourbits _d_o_t_ com>
 # GPL licensed (see end of file) * Use at your own risk!
@@ -14,6 +14,7 @@ export NCDIR=/var/www/nextcloud
 export ncc=/usr/local/bin/ncc
 export NCPCFG=${NCPCFG:-etc/ncp.cfg}
 export ARCH="$(dpkg --print-architecture)"
+export DB_PREFIX="$(php -r 'include("/var/www/nextcloud/config/config.php"); echo $CONFIG['"'dbtableprefix'"'];' || echo 'oc_')"
 [[ "${ARCH}" =~ ^(armhf|arm)$ ]] && ARCH="armv7"
 [[ "${ARCH}" == "arm64" ]] && ARCH=aarch64
 [[ "${ARCH}" == "amd64" ]] && ARCH=x86_64
@@ -21,7 +22,7 @@ export ARCH="$(dpkg --print-architecture)"
 export SYSTEMD_PAGER=
 
 [[ -f "$NCPCFG" ]] || export NCPCFG=/usr/local/etc/ncp.cfg
-[[ -f "$NCPCFG" ]] || { echo "$NCPCFG not found" >2; exit 1; }
+[[ -f "$NCPCFG" ]] || { echo "$NCPCFG not found" >&2; exit 1; }
 
 if [[ "$(ps -p 1 --no-headers -o "%c")" == "systemd" ]] && ! [[ -d "/run/systemd/system" ]]
 then
@@ -156,8 +157,9 @@ function set-nc-domain()
   if is_ncp_activated && is_app_enabled notify_push; then
     ncc config:system:set trusted_proxies 11 --value="127.0.0.1"
     ncc config:system:set trusted_proxies 12 --value="::1"
-    ncc config:system:set trusted_proxies 13 --value="${domain}"
-    ncc config:system:set trusted_proxies 14 --value="$(dig +short "${domain}")"
+#    ncc config:system:set trusted_proxies 13 --value="${domain}"
+    local domain_ip="$(dig +short "${domain}")"
+    [[ -z "$domain_ip" ]] || ncc config:system:set trusted_proxies 14 --value="$(dig +short "${domain}")"
     sleep 5 # this seems to be required in the VM for some reason. We get `http2 error: protocol error` after ncp-upgrade-nc
     for try in {1..5}
     do
@@ -212,6 +214,30 @@ function find_app_param_num()
 
 }
 
+function get_app_params() {
+  local script="${1?}"
+  local cfg_file="${CFGDIR}/${script%.sh}.cfg"
+  [[ -f "$cfg_file" ]] && {
+    local cfg="$( cat "$cfg_file" )"
+    local param_count="$(jq ".params | length" <<<"$cfg")"
+    local i=0
+    local json="{"
+    while [[ $i -lt $param_count ]]
+    do
+      param_id="$(jq -r ".params[$i].id" <<<"$cfg")"
+      param_val="$(jq -r ".params[$i].value" <<<"$cfg")"
+      json="${json}"$'\n'"  \"${param_id}\": \"${param_val}\""
+      i=$((i+1))
+      [[ $i -lt $param_count ]] && json="${json},"
+    done
+    json="${json}"$'\n'"}"
+    echo "$json"
+    return 0
+  }
+
+  return 1
+}
+
 install_template() {
   local template="${1?}"
   local target="${2?}"
@@ -490,7 +516,54 @@ function check_distro()
 
 function nc_version()
 {
-  ncc status | grep "version:" | awk '{ print $3 }'
+  ncc status | grep "versionstring:" | awk '{ print $3 }'
+}
+
+function determine_nc_update_version() {
+  local current supported current_maj supported_maj versions next_version
+  current="${1?}"
+  supported="${2?}"
+  requested="${3:-latest}"
+
+  #CURRENT="$(ncc status | grep "versionstring:" | awk '{ print $3 }')"
+  current_maj="${current%%.*}"
+  requested_maj="${requested%%.*}"
+  supported_maj="${supported%%.*}"
+
+  # If valid version is requested -> direct update, don't consider anything else
+  if [[ "$requested" =~ ^[0-9]*.[0-9]*.[0-9]*$ ]]
+  then
+    if ! is_more_recent_than "${requested}" "${current}"
+    then
+      echo "$current"
+      return 0
+    elif [[ "$requested_maj" -le "$((current_maj + 1))" ]]
+    then
+      echo "$requested"
+      return 0
+    fi
+  fi
+
+  versions="$(curl -q -L -H "Accept: application/vnd.github+json" -H "X-GitHub-Api-Version: 2022-11-28" https://api.github.com/repos/nextcloud/server/releases?per_page=100 | jq -r '.[].tag_name' | grep -v -e 'rc.$' -e 'beta.$' | sort -V)"
+  next_version="$(grep "v${current_maj}." <<<"${versions}" | tail -n 1 | tr -d 'v')"
+  next_version_maj="${next_version%%.*}"
+  next_version_min="${next_version#*.}"
+  next_version_min="${next_version_min%%.*}"
+
+  if [[ "${next_version}" == "${current}" ]]
+  then
+
+    if [[ "${supported_maj}" -le "${current_maj}" ]]
+    then
+      echo "$current"
+      # No update available
+      return 0
+    fi
+
+    next_version="$(grep "$v$((current_maj + 1))." <<< "${versions}" | tail -n 1 | tr -d 'v')"
+  fi
+
+  [[ -z "${next_version}" ]] || echo -n "${next_version}"
 }
 
 function get_ip()
@@ -543,7 +616,7 @@ function notify_admin()
 {
   local header="$1"
   local msg="$2"
-  local admins=$(mysql -u root nextcloud -Nse "select uid from oc_group_user where gid='admin';")
+  local admins=$(mysql -u root nextcloud -Nse "select uid from ${DB_PREFIX}group_user where gid='admin';")
   [[ "${admins}" == "" ]] && { echo "admin user not found" >&2; return 0; }
   while read -r admin
   do

etc/library.test

@@ -0,0 +1,76 @@
+#!/bin/bash
+set -ex
+
+fail() {
+  echo "$@"
+  exit 1
+}
+
+#shellcheck source=./library.sh
+. "$(dirname "$0")/library.sh" ||:
+
+
+case=1
+
+current=27.0.1
+supported=27.0.8
+requested=""
+
+expected=27.1.11
+
+actual="$(determine_nc_update_version "$current" "$supported" "$requested")"
+
+[[ "$expected"  == "$actual" ]] || fail "C$case: unexpected nc target version: $actual (expected: $expected)"
+
+
+case=2
+
+# TODO: Set to 26.0.1
+current=26.0.13
+supported=27.0.8
+requested=""
+
+expected=27.1.11
+
+actual="$(determine_nc_update_version "$current" "$supported" "$requested")"
+
+[[ "$expected"  == "$actual" ]] || fail "C$case: unexpected nc target version: $actual (expected: $expected)"
+
+
+case=3
+
+current=26.0.1
+supported=27.0.8
+requested="27.1.3"
+
+expected=27.1.3
+
+actual="$(determine_nc_update_version "$current" "$supported" "$requested")"
+
+[[ "$expected"  == "$actual" ]] || fail "C$case: unexpected nc target version: $actual (expected: $expected)"
+
+
+case=4
+
+current=29.0.5
+supported=27.0.8
+requested=""
+
+expected=29.0.6
+
+actual="$(determine_nc_update_version "$current" "$supported" "$requested")"
+
+[[ "$expected"  == "$actual" ]] || fail "C$case: unexpected nc target version: $actual (expected: $expected)"
+
+
+case=5
+
+current=29.0.5
+supported=27.0.8
+requested="27.0.8"
+
+expected="29.0.5"
+
+actual="$(determine_nc_update_version "$current" "$supported" "$requested")"
+
+[[ "$expected"  == "$actual" ]] || fail "C$case: unexpected nc target version: $actual (expected: $expected)"

etc/ncp-config.d/dnsmasq.cfg

@@ -15,8 +15,8 @@
     {
       "id": "DOMAIN",
       "name": "Domain",
-      "value": "mycloud.ownyourbits.com",
-      "suggest": "mycloud.ownyourbits.com"
+      "value": "mycloud.nextcloudpi.com",
+      "suggest": "mycloud.nextcloudpi.com"
     },
     {
       "id": "DNSSERVER",

etc/ncp-config.d/l10n/nc-export-ncp/es.json

@@ -1,7 +1,7 @@
 {
     "translations": {
         "DIR": "Directorio",
-        "Export NextCloudPi configuration": "Exportar la configuración de NextCloudPi",
+        "Export NextCloudPi configuration": "Exportar la configuración de NextcloudPi",
         "nc-export-ncp": "nc-export-ncp"
     }
 }

etc/ncp-config.d/l10n/nc-export-ncp/pt.json

@@ -1,7 +1,7 @@
 {
     "translations": {
         "DIR": "Diretório",
-        "Export NextCloudPi configuration": "Exportar configurações do NextCloudPi",
+        "Export NextCloudPi configuration": "Exportar configurações do NextcloudPi",
         "nc-export-ncp": "nc-exportar-ncp"
     }
 }

etc/ncp-config.d/l10n/nc-export-ncp/zh.json

@@ -1,7 +1,7 @@
 {
     "translations": {
         "DIR": "位置",
-        "Export NextCloudPi configuration": "匯出NextCloudPi設定",
+        "Export NextCloudPi configuration": "匯出NextcloudPi設定",
         "nc-export-ncp": "匯出NCP設定"
     }
 }

etc/ncp-config.d/l10n/nc-forward-ports/es.json

@@ -1,6 +1,6 @@
 {
     "translations": {
-        "For NextCloudPi to be able to setup your ports, UPnP must be activated\nin your router. Activate it now on your router admin webpage.\n\n** UPnP is considered a security risk **\n\nDon't forget to disable it afterwards": "Para que NextCloudPi sea capaz de configurar tus puertos, UPnP debe estar activado\nen tu router. Actívalo ahora en la página de administración de tu router\n\n** UPnP se considera un riesgo de seguridad\n\nNo olvides deshabilitarlo más tarde",
+        "For NextCloudPi to be able to setup your ports, UPnP must be activated\nin your router. Activate it now on your router admin webpage.\n\n** UPnP is considered a security risk **\n\nDon't forget to disable it afterwards": "Para que NextcloudPi sea capaz de configurar tus puertos, UPnP debe estar activado\nen tu router. Actívalo ahora en la página de administración de tu router\n\n** UPnP se considera un riesgo de seguridad\n\nNo olvides deshabilitarlo más tarde",
         "HTTPPORT": "Puerto HTTP",
         "HTTPSPORT": "Puerto HTTPS",
         "Instructions for UPnP Port Forwarding": "Instrucciones para UPnP Port Forwarding",

etc/ncp-config.d/l10n/nc-forward-ports/pt.json

@@ -1,6 +1,6 @@
 {
     "translations": {
-        "For NextCloudPi to be able to setup your ports, UPnP must be activated\nin your router. Activate it now on your router admin webpage.\n\n** UPnP is considered a security risk **\n\nDon't forget to disable it afterwards": "Para o NextCloudPi conseguir configurar as portas, UPnp deve estar ativo\nno seu Roteador.A ativação é feita na pagina de Web administração do seu roteador\n\n**UPnP é considerado um risco de segurança**\n\nNão esqueça de desativá-lo depois.",
+        "For NextCloudPi to be able to setup your ports, UPnP must be activated\nin your router. Activate it now on your router admin webpage.\n\n** UPnP is considered a security risk **\n\nDon't forget to disable it afterwards": "Para o NextcloudPi conseguir configurar as portas, UPnp deve estar ativo\nno seu Roteador.A ativação é feita na pagina de Web administração do seu roteador\n\n**UPnP é considerado um risco de segurança**\n\nNão esqueça de desativá-lo depois.",
         "HTTPPORT": "Porta http",
         "HTTPSPORT": "Porta https",
         "nc-forward-ports": "nc-portas-encaminhadas (UPnP)",

etc/ncp-config.d/l10n/nc-forward-ports/zh.json

@@ -1,6 +1,6 @@
 {
     "translations": {
-        "For NextCloudPi to be able to setup your ports, UPnP must be activated\nin your router. Activate it now on your router admin webpage.\n\n** UPnP is considered a security risk **\n\nDon't forget to disable it afterwards": "如果要讓NextCloudPi自動設置您的Port，請在您的路由器開啟UPnP\n因鑒於安全性等問題，請在設定成功後關閉UPnP",
+        "For NextCloudPi to be able to setup your ports, UPnP must be activated\nin your router. Activate it now on your router admin webpage.\n\n** UPnP is considered a security risk **\n\nDon't forget to disable it afterwards": "如果要讓NextcloudPi自動設置您的Port，請在您的路由器開啟UPnP\n因鑒於安全性等問題，請在設定成功後關閉UPnP",
         "HTTPPORT": "http Port",
         "HTTPSPORT": "https Port",
         "nc-forward-ports": "端口轉發（UPnP）",

etc/ncp-config.d/l10n/nc-import-ncp/es.json

@@ -1,7 +1,7 @@
 {
     "translations": {
         "FILE": "Fichero",
-        "Import NextCloudPi configuration from file": "Importa la configuración de NextCloudPi desde un fichero",
+        "Import NextCloudPi configuration from file": "Importa la configuración de NextcloudPi desde un fichero",
         "nc-import-ncp": "nc-import-ncp"
     }
 }

etc/ncp-config.d/l10n/nc-import-ncp/pt.json

@@ -1,7 +1,7 @@
 {
     "translations": {
         "FILE": "Arquivo",
-        "Import NextCloudPi configuration from file": "Importar  a configuração do  NextCloudPi de um arquivo",
+        "Import NextCloudPi configuration from file": "Importar  a configuração do  NextcloudPi de um arquivo",
         "nc-import-ncp": "nc-importa-ncp"
     }
 }

etc/ncp-config.d/l10n/nc-import-ncp/zh.json

@@ -1,7 +1,7 @@
 {
     "translations": {
         "FILE": "檔案位置",
-        "Import NextCloudPi configuration from file": "匯入NextCloudPi設定",
+        "Import NextCloudPi configuration from file": "匯入NextcloudPi設定",
         "nc-import-ncp": "匯入NCP設定"
     }
 }

etc/ncp-config.d/l10n/nc-info/pt.json

@@ -1,6 +1,6 @@
 {
     "translations": {
-        "Print NextCloudPi system info": "Imprime informações do sistema NextCloudPi",
+        "Print NextCloudPi system info": "Imprime informações do sistema NextcloudPi",
         "nc-info": "nc-info"
     }
 }

etc/ncp-config.d/l10n/nc-info/zh.json

@@ -1,6 +1,6 @@
 {
     "translations": {
-        "Print NextCloudPi system info": "顯示NextCloudPi系統信息",
+        "Print NextCloudPi system info": "顯示NextcloudPi系統信息",
         "nc-info": "NCP系統訊息"
     }
 }

etc/ncp-config.d/l10n/nc-limits/es.json

@@ -1,6 +1,6 @@
 {
     "translations": {
-        "Configure system limits for NextCloudPi": "Configura los límites de sistema para NextCloudPi",
+        "Configure system limits for NextCloudPi": "Configura los límites de sistema para NextcloudPi",
         "MAXFILESIZE":"Tamaño máximo de archivo en subida",
         "MEMORYLIMIT": "Límite de memoria",
         "PHPTHREADS": "Hilos PHP",

etc/ncp-config.d/l10n/nc-limits/pt.json

@@ -1,6 +1,6 @@
 {
     "translations": {
-        "Configure system limits for NextCloudPi": "Configura os limites de sistema para NextCloudPi",
+        "Configure system limits for NextCloudPi": "Configura os limites de sistema para NextcloudPi",
         "MAXFILESIZE":"Tamanho máximo de arquivo para upload",
         "MEMORYLIMIT": "Limite  de memória",
         "PHPTHREADS": "THREADS PHP",

etc/ncp-config.d/l10n/nc-limits/zh.json

@@ -1,6 +1,6 @@
 {
     "translations": {
-        "Configure system limits for NextCloudPi": "設定NextCloudPi的系統限制",
+        "Configure system limits for NextCloudPi": "設定NextcloudPi的系統限制",
         "MAXFILESIZE": "最大文件大小（<=2G）",
         "MEMORYLIMIT": "內存限制",
         "PHPTHREADS": "PHP線程",

etc/ncp-config.d/l10n/nc-nextcloud/pt.json

@@ -6,7 +6,7 @@
     "MEMORYLIMIT":"Limíte de Memória",
     "MAXTRANSFERTIME":"Tempo máximo de transferência",
     "NextCloud installation":"Instalação NextCloud",
-    "This new installation will cleanup current\nNextCloud instance, including files and database.\n\nYou can later use nc-init to configure to NextCloudPi defaults\n\n** perform backup before proceding **\n\nYou can use nc-backup ":"Esta nova instalação irá apagar a instância atual\ndo NextCloud, incluindo os arquivos e o Banco de Dados.\nVocê depois pode usar o nc-init para configurar os padrões do NextCloudPi\n\n**Realize Backup antes de Continuar**\n\nVocê pode usar o nc-backup",
+    "This new installation will cleanup current\nNextCloud instance, including files and database.\n\nYou can later use nc-init to configure to NextCloudPi defaults\n\n** perform backup before proceding **\n\nYou can use nc-backup ":"Esta nova instalação irá apagar a instância atual\ndo NextCloud, incluindo os arquivos e o Banco de Dados.\nVocê depois pode usar o nc-init para configurar os padrões do NextcloudPi\n\n**Realize Backup antes de Continuar**\n\nVocê pode usar o nc-backup",
     "nc-nextcloud":"nc-nextcloud",
     "Install any NextCloud version":"Instalar qualquer versão NextCloud"
   }