[mysql-router][K8s] NodePort is not reachable after the biggest ordinal pod upgrade

Description

Hi,

Problem: the NodePort got silently changed (removed?) after the first unit mysql-router-k8s refresh.

See the picture:

STR (more components on screenshot, here is the minimal STR):

  • juju deploy mysql-k8s, mysql-router-k8s, data-integrator # all stable tracks.

  • juju run data-integrator/leader get-credentials => test connection using mysql client, ALL OK here

  • juju refresh mysql-router-k8s --channel 8.0/candidate => the first unit upgraded properly and waiting for resume-upgrade.

  • Test the previous connection => doesn’t work anymore, as NodePort has changed, but:

    • get-credentials still returns you the old port which is not reachable. Can we fix this?

    • using microk8s.kubectl get svc --all-namespaces -o go-template='{{range .items}}{{range.spec.ports}}{{if .nodePort}}{{.nodePort}}{{"\n"}}{{end}}{{end}}{{end}}' you can brute-force the new port and you will connect properly, but it is:

      • impossible to guess the new port

      • who will copy the new port to the external app? This probably will have to be branched to the independent ingress topic?

 

Q1: Can we avoid NodePort update on mysql-router-k8s refresh?

Q2: can it be that each mysql-router-k8s unit opens independent/separate NodePorts? 0_o (I suspect this, but need to explore it further)

Acceptance Criteria

None

Attachments

1

Activity

Show:

Shayan Patel November 14, 2024 at 12:36 PM

There is already a draft PR (PR #328) to create a dedicated service for external connectivity to router k8s. The remaining effort is to confirm that the issue is resolved once that PR is merged

paulo.machado September 2, 2024 at 12:16 PM

, yes! Having that will prevent juju resetting the service. The router will only need to ensure correct pod labeling for service selection

Carl Csaposs August 28, 2024 at 7:03 AM

re Q1: maybe we need to create a different service instead of using the one created by juju to avoid getting patched on refresh

Done
Pinned fields
Click on the next to a field label to start pinning.

Details

Priority

Assignee

Reporter

Components

Sprint

Original estimate

Time remaining

8h

Created August 27, 2024 at 6:08 PM
Updated November 27, 2024 at 1:19 AM
Resolved November 27, 2024 at 1:19 AM