Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

[MySQL][VM]]K8s] Cannot start 10+ member of the cluster "Waiting to join the cluster"

Description

Hi,

STR:

  1. juju add-model mysql

  2. juju deploy ch:mysql --channel edge -n 15

Actual result: 9 nodes joined the cluster. Nodes 10-15 stuck in “Waiting to join the cluster“.
Expected result:

  • some indication, e.g. “hot-swap (max 10 members of cluster)“

  • removing some “active“ nodes, should pickup one from “waiting“ state

None of above are true. After removing 3 active nodes + “jhack ffwd“, error:

ERROR unit.mysql/4.juju-log Failed to get member state: mysqld daemon is down or unaccessible

See the picture:

NOTE: the further “add-unit mysql“ does join the cluster (id 15 and 16), but “waiting“ nodes stuck in waiting state (id 4,5,6,7,8,9):

P.S. The similar issue for PostgreSQL is

Acceptance Criteria

None

Attachments

4

Activity

Show:

paulo.machado May 16, 2023 at 9:10 PM

looking like this now:

Alex Lutay April 20, 2023 at 1:21 PM

ad discussed, let’s put unit in blocked state if peer relation unit-count says 9 units in cluster already.

Consider to re-check join cluster possibility on update-status hook and join if less then nine units in cluster otherwise stay blocked. P.S. Prevent DDOS for main DB from hot-swap units.

Alex Lutay March 6, 2023 at 10:20 AM

what is your design decision here?

  1. we will document 9 active mysql cluster nodes max

  2. IMHO, adding 10+ MySQL node should stay in waiting state until some currently active node gone with reasonable message (cluster max nodes 9 reached). Kind of hot-swap spare nodes.

OR

  • juju should not allow to start more then 9 nodes of application mysql. Can it be defined in metadata.yaml? Peer connections limit 9?

paulo.machado February 6, 2023 at 7:57 PM

, we know that mysql GR is limited to 9 nodes. Is that the case of making some kind of UI and documentation improvement to inform that?

Done
Pinned fields
Click on the next to a field label to start pinning.

Details

Priority

Assignee

Reporter

Components

Sprint

Original estimate

Time remaining

4h

Created November 21, 2022 at 11:34 AM
Updated June 15, 2023 at 12:54 PM
Resolved June 15, 2023 at 12:54 PM