2015.01.28

DRBDを用いたHA構成のChef Serverクラスタのプライマリ・セカンダリ移行 #getchef

Chef d-higuchi

この記事は1年以上前に投稿されました。情報が古い可能性がありますので、ご注意ください。

前回、DRBDを用いたHA構成のChef Serverクラスタを構築しました。
本稿では実例として、プライマリ・セカンダリのGraceful Transitions(ゆるやかな移行)を行ってみます。

■前提

環境

環境: QEMU+KVM仮想環境
パブリックネットワーク: 192.168.122.0/24
プライベートネットワーク: 192.168.100.0/24
フロントエンド: 1台
バックエンド: 2台
ロードバランサ: フロントエンド1台のため不要
名前解決: 全マシンの/etc/hostsに記載 (参考:Hostnames)
ファイアウォール: テストのため全解放 (参考:Firewalls)
インストールしたChef Server: chef-server-core-12.0.1-1.x86_64.rpm

フロントエンド

IPアドレス: 192.168.122.101
ホスト名: fe1.example.jp
OS: RHEL6.5
メモリ: 4GB
ディスク: 10GB (qcow2)

フロントエンドの共有ホスト名: chef-server.example.jp

バックエンド

パブリックIPアドレス: 192.168.122.111
プライベートIPアドレス: 192.168.100.111
ホスト名: be1.example.jp
OS: RHEL6.5
メモリ: 8GB
ディスク: 10GB (qcow2) + 5GB (qcow2)

パブリックIPアドレス: 192.168.122.112
プライベートIPアドレス: 192.168.100.112
ホスト名: be2.example.jp
OS: RHEL6.5
メモリ: 8GB
ディスク: 10GB (qcow2) + 5GB (qcow2)

共有する仮想IPアドレス: 192.168.122.110
ホスト名: be.example.jp

[redhat@be1 ~]$ sudo chef-server-ctl status run: bookshelf: (pid 1636) 201s, normally down; run: log: (pid 697) 233s run: keepalived: (pid 711) 233s; run: log: (pid 704) 233s run: nginx: (pid 1672) 200s, normally down; run: log: (pid 703) 233s run: oc_bifrost: (pid 1682) 198s, normally down; run: log: (pid 699) 233s run: oc_id: (pid 1712) 196s, normally down; run: log: (pid 705) 233s run: opscode-erchef: (pid 1723) 194s, normally down; run: log: (pid 700) 233s run: opscode-expander: (pid 1740) 192s, normally down; run: log: (pid 706) 233s run: opscode-expander-reindexer: (pid 1758) 189s, normally down; run: log: (pid 698) 233s run: opscode-solr4: (pid 1782) 187s, normally down; run: log: (pid 707) 233s run: postgresql: (pid 1805) 185s, normally down; run: log: (pid 702) 233s run: rabbitmq: (pid 1830) 184s, normally down; run: log: (pid 709) 233s run: redis_lb: (pid 2146) 168s, normally down; run: log: (pid 701) 233s [redhat@be1 ~]$

全サービスが起動しています。

[redhat@be1 ~]$ sudo chef-server-ctl ha-status [OK] keepalived HA services enabled. [OK] cluster status = master [OK] found VIP IP address and I am master [OK] found VRRP communications interface eth0 [OK] bookshelf is running correctly, and I am master. [OK] keepalived is running. [OK] nginx is running correctly, and I am master. [OK] oc_bifrost is running correctly, and I am master. [OK] oc_id is running correctly, and I am master. [OK] opscode-chef-mover is running. [OK] opscode-erchef is running correctly, and I am master. [OK] opscode-expander is running correctly, and I am master. [OK] opscode-expander-reindexer is running correctly, and I am master. [OK] opscode-solr4 is running correctly, and I am master. [OK] postgresql is running correctly, and I am master. [OK] rabbitmq is running correctly, and I am master. [OK] redis_lb is running correctly, and I am master.

[OK] all checks passed.

[redhat@be1 ~]$

状態はマスター(cluster status = master)で、各サービスが動作(is running)しており、マスター(I am master)となっています。

[redhat@be1 ~]$ cat /proc/drbd version: 8.4.5 (api:1/proto:86-101) GIT-hash: 1d360bde0e095d495786eaeb2a1ac76888e4db96 build by mockbuild@Build64R6, 2014-08-17 19:26:04 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r----- ns:109744 nr:0 dw:39616 dr:110957 al:18 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0 [redhat@be1 ~]$

[redhat@be1 ~]$ drbd-overview 0:pc0/0 Connected Primary/Secondary UpToDate/UpToDate /var/opt/opscode/drbd/data ext4 4.0G 160M 3.6G 5% [redhat@be1 ~]$

DRBD は対向ノードと接続(Connected)しており、プライマリロール(Primary)で動作し、データが一致して最新の状態(UpToDate)です。

be2.example.jp (セカンダリ)

[redhat@be2 ~]$ sudo chef-server-ctl status down: bookshelf: 137s; run: log: (pid 697) 137s run: keepalived: (pid 704) 137s; run: log: (pid 692) 137s down: nginx: 137s; run: log: (pid 699) 137s down: oc_bifrost: 137s; run: log: (pid 702) 137s down: oc_id: 137s; run: log: (pid 693) 137s down: opscode-erchef: 137s; run: log: (pid 701) 137s down: opscode-expander: 137s; run: log: (pid 703) 137s down: opscode-expander-reindexer: 137s; run: log: (pid 694) 137s down: opscode-solr4: 137s; run: log: (pid 695) 137s down: postgresql: 137s; run: log: (pid 698) 137s down: rabbitmq: 137s; run: log: (pid 696) 137s down: redis_lb: 137s; run: log: (pid 700) 137s [redhat@be2 ~]$

クラスタの状態を管理する keepalived 以外のサービスが停止しています。

[redhat@be2 ~]$ sudo chef-server-ctl ha-status [OK] keepalived HA services enabled. [OK] cluster status = backup [OK] did not find VIP IP address and I am not master [OK] found VRRP communications interface eth0 [OK] bookshelf is not running, and I am not master. [OK] keepalived is running. [OK] nginx is not running, and I am not master. [OK] oc_bifrost is not running, and I am not master. [OK] oc_id is not running, and I am not master. [OK] opscode-erchef is not running, and I am not master. [OK] opscode-expander is not running, and I am not master. [OK] opscode-expander-reindexer is not running, and I am not master. [OK] opscode-solr4 is not running, and I am not master. [OK] postgresql is not running, and I am not master. [OK] rabbitmq is not running, and I am not master. [OK] redis_lb is not running, and I am not master.

[OK] all checks passed.

[redhat@be2 ~]$

状態はバックアップ(cluster status = backup)で、keepalived 以外の各サービスは停止(is not running)しており、スレーブ(I am not master)となっています。ぱっと見でnotがわかりづらいので注意してください。

[redhat@be2 ~]$ cat /proc/drbd version: 8.4.5 (api:1/proto:86-101) GIT-hash: 1d360bde0e095d495786eaeb2a1ac76888e4db96 build by mockbuild@Build64R6, 2014-08-17 19:26:04 0: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r----- ns:0 nr:109744 dw:109744 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0 [redhat@be2 ~]$

[redhat@be2 ~]$ drbd-overview 0:pc0/0 Connected Secondary/Primary UpToDate/UpToDate [redhat@be2 ~]$

DRBD は対向ノードと接続(Connected)しており、セカンダリロール(Secondary)で動作し、データが一致していて最新の状態(UpToDate)です。

データの確認

ここではアップロード済の Cookbook だけを簡単に確認しておきます。

[redhat@ws chef-repo]$ knife cookbook list apt 2.6.1 bluepill 2.3.1 build-essential 2.1.3 nginx 2.7.4 ohai 2.0.1 rsyslog 1.13.0 runit 1.5.12 starter 1.0.0 yum 3.5.2 yum-epel 0.6.0 [redhat@ws chef-repo]$

■ゆるやかな移行

では、ゆるやかな移行 (Graceful Transitions) を行ってみましょう。特に障害が起こっていない状況での移行です。

be1.example.jpをプライマリからセカンダリに降格

現在プライマリである be1.example.jp の keepalived を停止します。

[redhat@be1 ~]$ sudo chef-server-ctl stop keepalived ok: down: keepalived: 0s, normally up [redhat@be1 ~]$

keepalived を停止するとフェイルオーバーが始まります。現プライマリでは、

仮想 IP アドレスを削除
すべてのサービスの停止
DRBD のアンマウント
DRBD のセカンダリ移行

が行われ、現セカンダリではその逆が行われるので、しばらく待ちます。
状況は chef-server-ctl status コマンドや chef-server-ctl ha-status コマンドで確認できます。

移行が完了すると、

[redhat@be1 ~]$ sudo chef-server-ctl status down: bookshelf: 25s; run: log: (pid 697) 679s down: keepalived: 44s, normally up; run: log: (pid 704) 679s down: nginx: 28s; run: log: (pid 703) 680s down: oc_bifrost: 29s; run: log: (pid 699) 680s down: oc_id: 31s; run: log: (pid 705) 680s down: opscode-erchef: 33s; run: log: (pid 700) 680s down: opscode-expander: 35s; run: log: (pid 706) 680s down: opscode-expander-reindexer: 37s; run: log: (pid 698) 680s down: opscode-solr4: 38s; run: log: (pid 707) 680s down: postgresql: 41s; run: log: (pid 702) 680s down: rabbitmq: 42s; run: log: (pid 709) 680s down: redis_lb: 44s; run: log: (pid 701) 680s [redhat@be1 ~]$

サービスがすべて停止し、

[redhat@be1 ~]$ sudo chef-server-ctl ha-status [OK] keepalived HA services enabled. [OK] cluster status = backup [OK] did not find VIP IP address and I am not master [OK] found VRRP communications interface eth0 [OK] bookshelf is not running, and I am not master. [ERROR] keepalived is not running. [OK] nginx is not running, and I am not master. [OK] oc_bifrost is not running, and I am not master. [OK] oc_id is not running, and I am not master. [OK] opscode-chef-mover is running. [OK] opscode-erchef is not running, and I am not master. [OK] opscode-expander is not running, and I am not master. [OK] opscode-expander-reindexer is not running, and I am not master. [OK] opscode-solr4 is not running, and I am not master. [OK] postgresql is not running, and I am not master. [OK] rabbitmq is not running, and I am not master. [OK] redis_lb is not running, and I am not master.

[ERROR] ERRORS WERE DETECTED.

[redhat@be1 ~]$

状態がバックアップ(cluster status = backup)となります。

[redhat@be1 ~]$ cat /proc/drbd version: 8.4.5 (api:1/proto:86-101) GIT-hash: 1d360bde0e095d495786eaeb2a1ac76888e4db96 build by mockbuild@Build64R6, 2014-08-17 19:26:04 0: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r----- ns:108304 nr:28516 dw:58996 dr:84705 al:17 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0 [redhat@be1 ~]$

[redhat@be1 ~]$ drbd-overview 0:pc0/0 Connected Secondary/Primary UpToDate/UpToDate [redhat@be1 ~]$

DRBD もセカンダリに降格し、対向ノードはプライマリに昇格しました。

プライマリに昇格した be2.example.jp の確認

be2.example.jp の状態も同様に確認してみます。

[redhat@be2 ~]$ sudo chef-server-ctl status run: bookshelf: (pid 1902) 144s, normally down; run: log: (pid 697) 798s run: keepalived: (pid 704) 798s; run: log: (pid 692) 798s run: nginx: (pid 1937) 143s, normally down; run: log: (pid 699) 798s run: oc_bifrost: (pid 1947) 142s, normally down; run: log: (pid 702) 798s run: oc_id: (pid 1978) 140s, normally down; run: log: (pid 693) 798s run: opscode-erchef: (pid 1989) 137s, normally down; run: log: (pid 701) 798s run: opscode-expander: (pid 2006) 135s, normally down; run: log: (pid 703) 798s run: opscode-expander-reindexer: (pid 2025) 133s, normally down; run: log: (pid 694) 798s run: opscode-solr4: (pid 2049) 130s, normally down; run: log: (pid 695) 798s run: postgresql: (pid 2074) 129s, normally down; run: log: (pid 698) 798s run: rabbitmq: (pid 2097) 127s, normally down; run: log: (pid 696) 798s run: redis_lb: (pid 2419) 111s, normally down; run: log: (pid 700) 798s [redhat@be2 ~]$

サービスがすべて起動し、

[redhat@be2 ~]$ sudo chef-server-ctl ha-status [OK] keepalived HA services enabled. [OK] cluster status = master [OK] found VIP IP address and I am master [OK] found VRRP communications interface eth0 [OK] bookshelf is running correctly, and I am master. [OK] keepalived is running. [OK] nginx is running correctly, and I am master. [OK] oc_bifrost is running correctly, and I am master. [OK] oc_id is running correctly, and I am master. [OK] opscode-erchef is running correctly, and I am master. [OK] opscode-expander is running correctly, and I am master. [OK] opscode-expander-reindexer is running correctly, and I am master. [OK] opscode-solr4 is running correctly, and I am master. [OK] postgresql is running correctly, and I am master. [OK] rabbitmq is running correctly, and I am master. [OK] redis_lb is running correctly, and I am master.

[OK] all checks passed.

[redhat@be2 ~]$

状態がマスター(cluster status = master)となっています。

[redhat@be2 ~]$ cat /proc/drbd version: 8.4.5 (api:1/proto:86-101) GIT-hash: 1d360bde0e095d495786eaeb2a1ac76888e4db96 build by mockbuild@Build64R6, 2014-08-17 19:26:04 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r----- ns:29184 nr:108304 dw:137488 dr:4849 al:16 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0 [redhat@be2 ~]$

[redhat@be2 ~]$ drbd-overview 0:pc0/0 Connected Primary/Secondary UpToDate/UpToDate /var/opt/opscode/drbd/data ext4 4.0G 160M 3.6G 5% [redhat@be2 ~]$

DRBD もプライマリに昇格しています。

be1.example.jp のセカンダリ化

keepalived を起動してセカンダリとしてクラスタに復帰させます。

[redhat@be1 ~]$ sudo chef-server-ctl start keepalived ok: run: keepalived: (pid 3196) 0s [redhat@be1 ~]$

[redhat@be1 ~]$ sudo chef-server-ctl status down: bookshelf: 409s; run: log: (pid 697) 1063s run: keepalived: (pid 3196) 19s; run: log: (pid 704) 1063s down: nginx: 411s; run: log: (pid 703) 1063s down: oc_bifrost: 412s; run: log: (pid 699) 1063s down: oc_id: 414s; run: log: (pid 705) 1063s down: opscode-erchef: 416s; run: log: (pid 700) 1063s down: opscode-expander: 418s; run: log: (pid 706) 1063s down: opscode-expander-reindexer: 420s; run: log: (pid 698) 1063s down: opscode-solr4: 421s; run: log: (pid 707) 1063s down: postgresql: 424s; run: log: (pid 702) 1063s down: rabbitmq: 425s; run: log: (pid 709) 1063s down: redis_lb: 427s; run: log: (pid 701) 1063s [redhat@be1 ~]$

[redhat@be1 ~]$ sudo chef-server-ctl ha-status [OK] keepalived HA services enabled. [OK] cluster status = backup [OK] did not find VIP IP address and I am not master [OK] found VRRP communications interface eth0 [OK] bookshelf is not running, and I am not master. [OK] keepalived is running. [OK] nginx is not running, and I am not master. [OK] oc_bifrost is not running, and I am not master. [OK] oc_id is not running, and I am not master. [OK] opscode-chef-mover is running. [OK] opscode-erchef is not running, and I am not master. [OK] opscode-expander is not running, and I am not master. [OK] opscode-expander-reindexer is not running, and I am not master. [OK] opscode-solr4 is not running, and I am not master. [OK] postgresql is not running, and I am not master. [OK] rabbitmq is not running, and I am not master. [OK] redis_lb is not running, and I am not master.

[OK] all checks passed.

[redhat@be1 ~]$

keepalived を起動したからといって勝手にマスターになったりはしません。

データの確認

事前に確認しておいたアップロード済 Cookbook のリストが同じであることを確認します。

新たに Cookbook をアップロードします。

[redhat@ws chef-repo]$ knife upload cookbooks/apache2 Created cookbooks/apache2 [redhat@ws chef-repo]$

Cookbook がアップロードできたことを確認します。

[redhat@ws chef-repo]$ knife cookbook list apache2 3.0.0 apt 2.6.1 bluepill 2.3.1 build-essential 2.1.3 nginx 2.7.4 ohai 2.0.1 rsyslog 1.13.0 runit 1.5.12 starter 1.0.0 yum 3.5.2 yum-epel 0.6.0 [redhat@ws chef-repo]$

簡単ですが、ゆるやかな移行の成功を確認できたとしましょう。

■再度ゆるやかな移行を行う

先の移行では be1(プライマリ),be2(セカンダリ) を be1(セカンダリ),be2(プライマリ) としました。
ここでは再度、ゆるやかな移行を行ってみます。

be2.example.jpをプライマリからセカンダリに降格

現在プライマリである be2.example.jp の keepalived を停止します。

[redhat@be2 ~]$ sudo chef-server-ctl stop keepalived ok: down: keepalived: 0s, normally up [redhat@be2 ~]$

フェイルオーバーが始まるので、しばらく待ちます。移行が完了すると、

[redhat@be2 ~]$ sudo chef-server-ctl status down: bookshelf: 52s; run: log: (pid 697) 1825s down: keepalived: 70s, normally up; run: log: (pid 692) 1825s down: nginx: 53s; run: log: (pid 699) 1825s down: oc_bifrost: 54s; run: log: (pid 702) 1825s down: oc_id: 56s; run: log: (pid 693) 1825s down: opscode-erchef: 57s; run: log: (pid 701) 1825s down: opscode-expander: 59s; run: log: (pid 703) 1825s down: opscode-expander-reindexer: 61s; run: log: (pid 694) 1825s down: opscode-solr4: 63s; run: log: (pid 695) 1825s down: postgresql: 66s; run: log: (pid 698) 1825s down: rabbitmq: 67s; run: log: (pid 696) 1825s down: redis_lb: 69s; run: log: (pid 700) 1825s [redhat@be2 ~]$

サービスがすべて停止し、

[redhat@be2 ~]$ sudo chef-server-ctl ha-status [OK] keepalived HA services enabled. [OK] cluster status = backup [OK] did not find VIP IP address and I am not master [OK] found VRRP communications interface eth0 [OK] bookshelf is not running, and I am not master. [ERROR] keepalived is not running. [OK] nginx is not running, and I am not master. [OK] oc_bifrost is not running, and I am not master. [OK] oc_id is not running, and I am not master. [OK] opscode-erchef is not running, and I am not master. [OK] opscode-expander is not running, and I am not master. [OK] opscode-expander-reindexer is not running, and I am not master. [OK] opscode-solr4 is not running, and I am not master. [OK] postgresql is not running, and I am not master. [OK] rabbitmq is not running, and I am not master. [OK] redis_lb is not running, and I am not master.

[ERROR] ERRORS WERE DETECTED.

[redhat@be2 ~]$

状態がバックアップ(cluster status = backup)となります。

[redhat@be2 ~]$ cat /proc/drbd version: 8.4.5 (api:1/proto:86-101) GIT-hash: 1d360bde0e095d495786eaeb2a1ac76888e4db96 build by mockbuild@Build64R6, 2014-08-17 19:26:04 0: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r----- ns:49952 nr:136900 dw:186852 dr:8797 al:23 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0 [redhat@be2 ~]$

[redhat@be2 ~]$ drbd-overview 0:pc0/0 Connected Secondary/Primary UpToDate/UpToDate [redhat@be2 ~]$

DRBD もセカンダリに降格し、対向ノードはプライマリに昇格しました。

プライマリに昇格した be1.example.jp の確認

be1.example.jp の状態も同様に確認してみます。

[redhat@be1 ~]$ sudo chef-server-ctl status run: bookshelf: (pid 3605) 174s, normally down; run: log: (pid 697) 1951s run: keepalived: (pid 3196) 907s; run: log: (pid 704) 1951s run: nginx: (pid 3640) 172s, normally down; run: log: (pid 703) 1951s run: oc_bifrost: (pid 3650) 171s, normally down; run: log: (pid 699) 1951s run: oc_id: (pid 3685) 168s, normally down; run: log: (pid 705) 1951s run: opscode-erchef: (pid 3744) 165s, normally down; run: log: (pid 700) 1951s run: opscode-expander: (pid 3810) 162s, normally down; run: log: (pid 706) 1951s run: opscode-expander-reindexer: (pid 3883) 159s, normally down; run: log: (pid 698) 1951s run: opscode-solr4: (pid 3956) 156s, normally down; run: log: (pid 707) 1951s run: postgresql: (pid 3991) 153s, normally down; run: log: (pid 702) 1951s run: rabbitmq: (pid 4064) 152s, normally down; run: log: (pid 709) 1951s run: redis_lb: (pid 4531) 133s, normally down; run: log: (pid 701) 1951s [redhat@be1 ~]$

サービスがすべて起動し、

[OK] all checks passed.

[redhat@be1 ~]$

状態がマスター(cluster status = master)となっています。

[redhat@be1 ~]$ cat /proc/drbd version: 8.4.5 (api:1/proto:86-101) GIT-hash: 1d360bde0e095d495786eaeb2a1ac76888e4db96 build by mockbuild@Build64R6, 2014-08-17 19:26:04 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r----- ns:137512 nr:49952 dw:109640 dr:91950 al:19 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0 [redhat@be1 ~]$

[redhat@be1 ~]$ drbd-overview 0:pc0/0 Connected Primary/Secondary UpToDate/UpToDate /var/opt/opscode/drbd/data ext4 4.0G 164M 3.6G 5% [redhat@be1 ~]$

DRBD もプライマリに昇格しています。

be2.example.jp のセカンダリ化

keepalived を起動してセカンダリとしてクラスタに復帰させます。

[redhat@be2 ~]$ sudo chef-server-ctl start keepalived ok: run: keepalived: (pid 3380) 0s [redhat@be2 ~]$

[redhat@be2 ~]$ sudo chef-server-ctl status down: bookshelf: 351s; run: log: (pid 697) 2124s run: keepalived: (pid 3380) 58s; run: log: (pid 692) 2124s down: nginx: 352s; run: log: (pid 699) 2124s down: oc_bifrost: 353s; run: log: (pid 702) 2124s down: oc_id: 355s; run: log: (pid 693) 2124s down: opscode-erchef: 356s; run: log: (pid 701) 2124s down: opscode-expander: 358s; run: log: (pid 703) 2124s down: opscode-expander-reindexer: 360s; run: log: (pid 694) 2124s down: opscode-solr4: 362s; run: log: (pid 695) 2124s down: postgresql: 365s; run: log: (pid 698) 2124s down: rabbitmq: 366s; run: log: (pid 696) 2124s down: redis_lb: 368s; run: log: (pid 700) 2124s [redhat@be2 ~]$

[OK] all checks passed.

[redhat@be2 ~]$

[redhat@be2 ~]$ cat /proc/drbd version: 8.4.5 (api:1/proto:86-101) GIT-hash: 1d360bde0e095d495786eaeb2a1ac76888e4db96 build by mockbuild@Build64R6, 2014-08-17 19:26:04 0: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r----- ns:49952 nr:138108 dw:188060 dr:8797 al:23 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0 [redhat@be2 ~]$

[redhat@be2 ~]$ drbd-overview 0:pc0/0 Connected Secondary/Primary UpToDate/UpToDate [redhat@be2 ~]$

keepalived を起動したからといって勝手にマスターになったりはしません。

データの確認

移行前にアップロードした Cookbook が確かに存在していることを確認します。

■まとめ

本稿では、DRBDを用いたHA構成のChef Serverクラスタにおけるプライマリ・セカンダリのゆるやかな移行を行ってみました。特に障害が起こっていない状態であるため、何も問題はありませんでしたが、普段の何も起きていない状態でどのような処理が行われるかや手順の確認を行い、心構えをしておくとよいでしょう。