Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

[MySQL][K8s] Uncaught exception ops.pebble.ConnectionError: Could not connect to Pebble: socket not found at '/charm/containers/mysql/pebble.socket' (container restarted?)

Description

Hi,

Noticed by on testing 8.0/candidate (rev 150):

unit-db1-0: 15:14:51 DEBUG unit.db1/0.juju-log ops 2.14.0 up and running. 15:55:56 [290/814] unit-db1-0: 15:14:51 DEBUG unit.db1/0.juju-log no relation on 'tracing': tracing not ready unit-db1-0: 15:14:51 DEBUG unit.db1/0.juju-log <class '__main__.MySQLOperatorCharm'>.<property object at 0x7d43d47ec950> returned None; quietly disabling charm_tracing for the run. unit-db1-0: 15:14:51 DEBUG unit.db1/0.juju-log Emitting Juju event config_changed. unit-db1-0: 15:14:52 ERROR unit.db1/0.juju-log Uncaught exception while in charm code: Traceback (most recent call last): File "/var/lib/juju/agents/unit-db1-0/charm/./src/charm.py", line 828, in <module> main(MySQLOperatorCharm) File "/var/lib/juju/agents/unit-db1-0/charm/venv/ops/main.py", line 548, in main manager.run() File "/var/lib/juju/agents/unit-db1-0/charm/venv/ops/main.py", line 527, in run self._emit() File "/var/lib/juju/agents/unit-db1-0/charm/venv/ops/main.py", line 516, in _emit _emit_charm_event(self.charm, self.dispatcher.event_name) File "/var/lib/juju/agents/unit-db1-0/charm/venv/ops/main.py", line 147, in _emit_charm_event event_to_emit.emit(*args, **kwargs) File "/var/lib/juju/agents/unit-db1-0/charm/venv/ops/framework.py", line 348, in emit framework._emit(event) File "/var/lib/juju/agents/unit-db1-0/charm/venv/ops/framework.py", line 860, in _emit self._reemit(event_path) File "/var/lib/juju/agents/unit-db1-0/charm/venv/ops/framework.py", line 950, in _reemit custom_handler(event) File "/var/lib/juju/agents/unit-db1-0/charm/lib/charms/tempo_k8s/v1/charm_tracing.py", line 544, in wrapped_function return callable(*args, **kwargs) # type: ignore File "/var/lib/juju/agents/unit-db1-0/charm/./src/charm.py", line 466, in _on_config_changed config_content = self._mysql.read_file_content(MYSQLD_CONFIG_FILE) File "/var/lib/juju/agents/unit-db1-0/charm/lib/charms/tempo_k8s/v1/charm_tracing.py", line 544, in wrapped_function return callable(*args, **kwargs) # type: ignore File "/var/lib/juju/agents/unit-db1-0/charm/src/mysql_k8s_helpers.py", line 677, in read_file_content if not self.container.exists(path): File "/var/lib/juju/agents/unit-db1-0/charm/venv/ops/model.py", line 2678, in exists self._pebble.list_files(str(path), itself=True) File "/var/lib/juju/agents/unit-db1-0/charm/venv/ops/pebble.py", line 2464, in list_files resp = self._request('GET', '/v1/files', query) File "/var/lib/juju/agents/unit-db1-0/charm/venv/ops/pebble.py", line 1859, in _request response = self._request_raw(method, path, query, headers, data) File "/var/lib/juju/agents/unit-db1-0/charm/venv/ops/pebble.py", line 1912, in _request_raw raise ConnectionError( ops.pebble.ConnectionError: Could not connect to Pebble: socket not found at '/charm/containers/mysql/pebble.socket' (container restarted?) unit-db1-0: 15:14:52 ERROR juju.worker.uniter.operation hook "config-changed" (via hook dispatching script: dispatch) failed: exit status 1

STR:

sudo lxc launch ubuntu:22.04 mysql --vm -c limits.cpu=4 -c limits.memory=16GiB -d root,size=32GiB sudo lxc exec mysql bash su -l ubuntu sudo apt -y update && sudo apt -y upgrade sudo snap install juju --channel=3.4/stable # 3.4.3 mkdir -p /home/ubuntu/.local/share sudo snap install microk8s --channel=1.30-strict/stable sudo usermod -a -G snap_microk8s $USER sudo chown -f -R $USER ~/.kube newgrp snap_microk8s sudo microk8s.enable hostpath-storage dns rbac microk8s status --wait-ready juju bootstrap microk8s micro juju add-model rome juju deploy mysql-k8s db1 --trust --channel=8.0/candidate --config profile=testing --config cluster-name=rome --base ubuntu@22.04

AR: see the error above, auto-fixed for me in 6 seconds, auto-fixed for Mohamed in 5 minutes…

I can NOT reproduce this on MicroK8s 1.28-strict inside multipass,
but well reproducible in 1.28-strict, 1.29-strict and 1.30-strict for juju 3.4.3 and ops 2.14.0.

NOTE: it looks like this is reproducible for the very first time only. If you remove application and re-deploy - never happens for me. Also never happens after LXC restart… Crazy…

Acceptance Criteria

None

Activity

Show:

Alex Lutay June 26, 2024 at 7:36 PM

Quickly fixed by Paulo in revision 153. Resolved.

Done
Pinned fields
Click on the next to a field label to start pinning.

Details

Assignee

Reporter

Components

Sprint

Priority

More fields

Created June 26, 2024 at 3:58 PM
Updated June 26, 2024 at 7:36 PM
Resolved June 26, 2024 at 7:36 PM