오늘은 기존 단일시스템에서 ext4 convert가 아닌 RHCS환경에서의 PM을 통한 작업시나리오를 더불어 작성해볼까 한다.
일단 먼저 변환하기 위해서 아래와 같은 RPM이 필요하다.
그렇치 않으면 하기와 같은 메시지를 맞닿들일수 있다..
tune2fs 1.39 (29-May-2006)Invalid filesystem option set: extents,uninit_bg,dir_indexIn Red Hat Enterprise Linux 5, ext4 support comes with the additional package 'e4fsprogs'. So in order to convert an ext3 file system to ext4, use the command 'tune4fs' from 'e4fsprogs'.
- rhel5환경에서는 ext4를 지원하기위해서는 추가적인 RPM이 필요하다고 하네요. 그런이유는 변환하기 위해서다.
[root@ ]# yum install e4fsprogs
Loaded plugins: downloadonly, katello, product-id, security, subscription-
: manager
Updating certificate-based repositories.
Repository 'munin' is missing name in configuration, using id
Unable to read consumer identity
munin | 951 B 00:00
rhel-Server | 1.5 kB 00:00
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package e4fsprogs.x86_64 0:1.41.12-2.el5 set to be updated
--> Finished Dependency Resolution
Dependencies Resolved
================================================================================
Package Arch Version Repository Size
================================================================================
Installing:
e4fsprogs x86_64 1.41.12-2.el5 rhel-Server 1.1 M
Transaction Summary
================================================================================
Install 1 Package(s)
Upgrade 0 Package(s)
Total download size: 1.1 M
Is this ok [y/N]: y
Downloading Packages:
e4fsprogs-1.41.12-2.el5.x86_64.rpm | 1.1 MB 00:00
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : e4fsprogs 1/1
Installed products updated.
Installed:
e4fsprogs.x86_64 0:1.41.12-2.el5
Complete!
자~ 설치가 완료되었음 아래와 같이 ext4파일시스템으로 변환해보자.
[root@ ]# tune4fs -O extents,uninit_bg,dir_index /dev/VG02/mysql_data
tune4fs 1.41.12 (17-May-2010)
Please run e4fsck on the filesystem.
얼마걸리지 않아 완료되었고 마지막에 파일시스템체크가 필요하다는 내용이 있다.
[root@ ]# e4fsck /dev/VG02/mysql_data
e4fsck 1.41.12 (17-May-2010)
One or more block group descriptor checksums are invalid. Fix<y>? yes
Group descriptor 0 checksum is invalid. FIXED.
Group descriptor 1 checksum is invalid. FIXED.
Group descriptor 2 checksum is invalid. FIXED.
Group descriptor 3 checksum is invalid. FIXED.
Group descriptor 4 checksum is invalid. FIXED.
Group descriptor 5 checksum is invalid. FIXED.
Group descriptor 6 checksum is invalid. FIXED.
Group descriptor 7 checksum is invalid. FIXED.
Group descriptor 8 checksum is invalid. FIXED.
Group descriptor 9 checksum is invalid. FIXED.
Group descriptor 10 checksum is invalid. FIXED.
Group descriptor 11 checksum is invalid. FIXED.
Group descriptor 12 checksum is invalid. FIXED.
Group descriptor 13 checksum is invalid. FIXED.
Group descriptor 14 checksum is invalid. FIXED.
Group descriptor 15 checksum is invalid. FIXED.
Group descriptor 16 checksum is invalid. FIXED.
Group descriptor 17 checksum is invalid. FIXED.
Group descriptor 18 checksum is invalid. FIXED.
Group descriptor 19 checksum is invalid. FIXED.
Group descriptor 20 checksum is invalid. FIXED.
Group descriptor 21 checksum is invalid. FIXED.
Group descriptor 22 checksum is invalid. FIXED.
Group descriptor 23 checksum is invalid. FIXED.
Group descriptor 24 checksum is invalid. FIXED.
Group descriptor 25 checksum is invalid. FIXED.
Group descriptor 26 checksum is invalid. FIXED.
Group descriptor 27 checksum is invalid. FIXED.
Group descriptor 28 checksum is invalid. FIXED.
Group descriptor 29 checksum is invalid. FIXED.
Group descriptor 30 checksum is invalid. FIXED.
Group descriptor 31 checksum is invalid. FIXED.
Group descriptor 32 checksum is invalid. FIXED.
Group descriptor 33 checksum is invalid. FIXED.
Group descriptor 34 checksum is invalid. FIXED.
Group descriptor 35 checksum is invalid. FIXED.
Group descriptor 36 checksum is invalid. FIXED.
Group descriptor 37 checksum is invalid. FIXED.
Group descriptor 38 checksum is invalid. FIXED.
Group descriptor 39 checksum is invalid. FIXED.
Group descriptor 40 checksum is invalid. FIXED.
Group descriptor 41 checksum is invalid. FIXED.
Group descriptor 42 checksum is invalid. FIXED.
Group descriptor 43 checksum is invalid. FIXED.
Group descriptor 44 checksum is invalid. FIXED.
Group descriptor 45 checksum is invalid. FIXED.
Group descriptor 46 checksum is invalid. FIXED.
Group descriptor 47 checksum is invalid. FIXED.
Group descriptor 48 checksum is invalid. FIXED.
Group descriptor 49 checksum is invalid. FIXED.
Group descriptor 50 checksum is invalid. FIXED.
Group descriptor 51 checksum is invalid. FIXED.
Group descriptor 52 checksum is invalid. FIXED.
Group descriptor 53 checksum is invalid. FIXED.
Group descriptor 54 checksum is invalid. FIXED.
Group descriptor 55 checksum is invalid. FIXED.
Group descriptor 56 checksum is invalid. FIXED.
Group descriptor 57 checksum is invalid. FIXED.
Group descriptor 58 checksum is invalid. FIXED.
Group descriptor 59 checksum is invalid. FIXED.
Group descriptor 60 checksum is invalid. FIXED.
Group descriptor 61 checksum is invalid. FIXED.
Group descriptor 62 checksum is invalid. FIXED.
Group descriptor 63 checksum is invalid. FIXED.
Group descriptor 64 checksum is invalid. FIXED.
Group descriptor 65 checksum is invalid. FIXED.
Group descriptor 66 checksum is invalid. FIXED.
Group descriptor 67 checksum is invalid. FIXED.
Group descriptor 68 checksum is invalid. FIXED.
Group descriptor 69 checksum is invalid. FIXED.
Group descriptor 70 checksum is invalid. FIXED.
Group descriptor 71 checksum is invalid. FIXED.
Group descriptor 72 checksum is invalid. FIXED.
Group descriptor 73 checksum is invalid. FIXED.
Group descriptor 74 checksum is invalid. FIXED.
Group descriptor 75 checksum is invalid. FIXED.
Group descriptor 76 checksum is invalid. FIXED.
Group descriptor 77 checksum is invalid. FIXED.
Group descriptor 78 checksum is invalid. FIXED.
Group descriptor 79 checksum is invalid. FIXED.
Group descriptor 80 checksum is invalid. FIXED.
Group descriptor 81 checksum is invalid. FIXED.
Group descriptor 82 checksum is invalid. FIXED.
Group descriptor 83 checksum is invalid. FIXED.
Group descriptor 84 checksum is invalid. FIXED.
Group descriptor 85 checksum is invalid. FIXED.
Group descriptor 86 checksum is invalid. FIXED.
Group descriptor 87 checksum is invalid. FIXED.
Group descriptor 88 checksum is invalid. FIXED.
Group descriptor 89 checksum is invalid. FIXED.
Group descriptor 90 checksum is invalid. FIXED.
Group descriptor 91 checksum is invalid. FIXED.
Group descriptor 92 checksum is invalid. FIXED.
Group descriptor 93 checksum is invalid. FIXED.
Group descriptor 94 checksum is invalid. FIXED.
Group descriptor 95 checksum is invalid. FIXED.
Group descriptor 96 checksum is invalid. FIXED.
Group descriptor 97 checksum is invalid. FIXED.
Group descriptor 98 checksum is invalid. FIXED.
Group descriptor 99 checksum is invalid. FIXED.
Group descriptor 100 checksum is invalid. FIXED.
Group descriptor 101 checksum is invalid. FIXED.
Group descriptor 102 checksum is invalid. FIXED.
Group descriptor 103 checksum is invalid. FIXED.
Group descriptor 104 checksum is invalid. FIXED.
Group descriptor 105 checksum is invalid. FIXED.
Group descriptor 106 checksum is invalid. FIXED.
Group descriptor 107 checksum is invalid. FIXED.
Group descriptor 108 checksum is invalid. FIXED.
Group descriptor 109 checksum is invalid. FIXED.
Group descriptor 110 checksum is invalid. FIXED.
Group descriptor 111 checksum is invalid. FIXED.
Group descriptor 112 checksum is invalid. FIXED.
Group descriptor 113 checksum is invalid. FIXED.
Group descriptor 114 checksum is invalid. FIXED.
Group descriptor 115 checksum is invalid. FIXED.
Group descriptor 116 checksum is invalid. FIXED.
Group descriptor 117 checksum is invalid. FIXED.
Group descriptor 118 checksum is invalid. FIXED.
Group descriptor 119 checksum is invalid. FIXED.
Group descriptor 120 checksum is invalid. FIXED.
Group descriptor 121 checksum is invalid. FIXED.
Group descriptor 122 checksum is invalid. FIXED.
Group descriptor 123 checksum is invalid. FIXED.
Group descriptor 124 checksum is invalid. FIXED.
Group descriptor 125 checksum is invalid. FIXED.
Group descriptor 126 checksum is invalid. FIXED.
Group descriptor 127 checksum is invalid. FIXED.
Group descriptor 128 checksum is invalid. FIXED.
Group descriptor 129 checksum is invalid. FIXED.
Group descriptor 130 checksum is invalid. FIXED.
Group descriptor 131 checksum is invalid. FIXED.
Group descriptor 132 checksum is invalid. FIXED.
Group descriptor 133 checksum is invalid. FIXED.
Group descriptor 134 checksum is invalid. FIXED.
Group descriptor 135 checksum is invalid. FIXED.
Group descriptor 136 checksum is invalid. FIXED.
Group descriptor 137 checksum is invalid. FIXED.
Group descriptor 138 checksum is invalid. FIXED.
Group descriptor 139 checksum is invalid. FIXED.
Group descriptor 140 checksum is invalid. FIXED.
Group descriptor 141 checksum is invalid. FIXED.
Group descriptor 142 checksum is invalid. FIXED.
Group descriptor 143 checksum is invalid. FIXED.
Group descriptor 144 checksum is invalid. FIXED.
Group descriptor 145 checksum is invalid. FIXED.
Group descriptor 146 checksum is invalid. FIXED.
Group descriptor 147 checksum is invalid. FIXED.
Group descriptor 148 checksum is invalid. FIXED.
Group descriptor 149 checksum is invalid. FIXED.
Group descriptor 150 checksum is invalid. FIXED.
Group descriptor 151 checksum is invalid. FIXED.
Group descriptor 152 checksum is invalid. FIXED.
Group descriptor 153 checksum is invalid. FIXED.
Group descriptor 154 checksum is invalid. FIXED.
Group descriptor 155 checksum is invalid. FIXED.
Group descriptor 156 checksum is invalid. FIXED.
Group descriptor 157 checksum is invalid. FIXED.
Group descriptor 158 checksum is invalid. FIXED.
Group descriptor 159 checksum is invalid. FIXED.
Group descriptor 160 checksum is invalid. FIXED.
Group descriptor 161 checksum is invalid. FIXED.
Group descriptor 162 checksum is invalid. FIXED.
Group descriptor 163 checksum is invalid. FIXED.
Group descriptor 164 checksum is invalid. FIXED.
Group descriptor 165 checksum is invalid. FIXED.
Group descriptor 166 checksum is invalid. FIXED.
Group descriptor 167 checksum is invalid. FIXED.
Group descriptor 168 checksum is invalid. FIXED.
Group descriptor 169 checksum is invalid. FIXED.
Group descriptor 170 checksum is invalid. FIXED.
Group descriptor 171 checksum is invalid. FIXED.
Group descriptor 172 checksum is invalid. FIXED.
Group descriptor 173 checksum is invalid. FIXED.
Group descriptor 174 checksum is invalid. FIXED.
Group descriptor 175 checksum is invalid. FIXED.
Group descriptor 176 checksum is invalid. FIXED.
Group descriptor 177 checksum is invalid. FIXED.
Group descriptor 178 checksum is invalid. FIXED.
Group descriptor 179 checksum is invalid. FIXED.
Group descriptor 180 checksum is invalid. FIXED.
Group descriptor 181 checksum is invalid. FIXED.
Group descriptor 182 checksum is invalid. FIXED.
Group descriptor 183 checksum is invalid. FIXED.
Group descriptor 184 checksum is invalid. FIXED.
Group descriptor 185 checksum is invalid. FIXED.
Group descriptor 186 checksum is invalid. FIXED.
Group descriptor 187 checksum is invalid. FIXED.
Group descriptor 188 checksum is invalid. FIXED.
Group descriptor 189 checksum is invalid. FIXED.
Group descriptor 190 checksum is invalid. FIXED.
Group descriptor 191 checksum is invalid. FIXED.
Group descriptor 192 checksum is invalid. FIXED.
Group descriptor 193 checksum is invalid. FIXED.
Group descriptor 194 checksum is invalid. FIXED.
Group descriptor 195 checksum is invalid. FIXED.
Group descriptor 196 checksum is invalid. FIXED.
Group descriptor 197 checksum is invalid. FIXED.
Group descriptor 198 checksum is invalid. FIXED.
Group descriptor 199 checksum is invalid. FIXED.
Group descriptor 200 checksum is invalid. FIXED.
Group descriptor 201 checksum is invalid. FIXED.
Group descriptor 202 checksum is invalid. FIXED.
Group descriptor 203 checksum is invalid. FIXED.
Group descriptor 204 checksum is invalid. FIXED.
Group descriptor 205 checksum is invalid. FIXED.
Group descriptor 206 checksum is invalid. FIXED.
Group descriptor 207 checksum is invalid. FIXED.
Group descriptor 208 checksum is invalid. FIXED.
Group descriptor 209 checksum is invalid. FIXED.
Group descriptor 210 checksum is invalid. FIXED.
Group descriptor 211 checksum is invalid. FIXED.
Group descriptor 212 checksum is invalid. FIXED.
Group descriptor 213 checksum is invalid. FIXED.
Group descriptor 214 checksum is invalid. FIXED.
Group descriptor 215 checksum is invalid. FIXED.
Group descriptor 216 checksum is invalid. FIXED.
Group descriptor 217 checksum is invalid. FIXED.
Group descriptor 218 checksum is invalid. FIXED.
Group descriptor 219 checksum is invalid. FIXED.
Group descriptor 220 checksum is invalid. FIXED.
Group descriptor 221 checksum is invalid. FIXED.
Group descriptor 222 checksum is invalid. FIXED.
Group descriptor 223 checksum is invalid. FIXED.
Group descriptor 224 checksum is invalid. FIXED.
Group descriptor 225 checksum is invalid. FIXED.
Group descriptor 226 checksum is invalid. FIXED.
Group descriptor 227 checksum is invalid. FIXED.
Group descriptor 228 checksum is invalid. FIXED.
Group descriptor 229 checksum is invalid. FIXED.
Group descriptor 230 checksum is invalid. FIXED.
Group descriptor 231 checksum is invalid. FIXED.
Group descriptor 232 checksum is invalid. FIXED.
Group descriptor 233 checksum is invalid. FIXED.
Group descriptor 234 checksum is invalid. FIXED.
Group descriptor 235 checksum is invalid. FIXED.
Group descriptor 236 checksum is invalid. FIXED.
Group descriptor 237 checksum is invalid. FIXED.
Group descriptor 238 checksum is invalid. FIXED.
Group descriptor 239 checksum is invalid. FIXED.
Adding dirhash hint to filesystem.
/dev/VG02/mysql_data contains a file system with errors, check forced.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/VG02/mysql_data: ***** FILE SYSTEM WAS MODIFIED *****
/dev/VG02/mysql_data: 112/3932160 files (5.4% non-contiguous), 1154325/7864320 blocks
파일시스템용량에 따라 메타데이타정보가 달라지기 때문에 시간은 각각 틀릴수 있다.
HA-LVM구성된 ext3를 ext4로 변환하고 클러스터정보도 변경하고 하기와 같이 테스트를 해본다
[root@ ]# rg_test test /etc/cluster/cluster.conf start service mysql-server
Running in test mode.
Starting mysql-server...
volume_list=["VG01", "@virt-node2"]
<notice> Activating VG02/mysql_data
Logical volume "mysql_data" changed
<notice> Making resilient : lvchange -ay VG02/mysql_data
/usr/share/cluster/lvm_by_lv.sh: line 46: [: too many arguments
<notice> Resilient command: lvchange -ay VG02/mysql_data --config devices{filter=["a|/dev/sdb1|","a|/dev/sdc1|","a|/dev/sdd1|","a|/dev/sde1|","a|/dev/sdf1|","a|/dev/sdg1|","a|/dev/vda8|","r|.*|"]}
Reading all physical volumes. This may take a while...
Found volume group "VG03" using metadata type lvm2
Found volume group "VG02" using metadata type lvm2
Found volume group "VG01" using metadata type lvm2
<info> mounting /dev/VG02/mysql_data on /MYSQL_DATA
<debug> mount -t ext4 /dev/VG02/mysql_data /MYSQL_DATA
<info> quotaopts =
<debug> Link for eth0: Detected
<info> Adding IPv4 address 115.144.181.43/26 to eth0
<debug> Pinging addr 115.144.181.43 from dev eth0
<debug> Sending gratuitous ARP: 115.144.181.43 52:54:00:a5:e5:d4 brd ff:ff:ff:ff:ff:ff
<info> Executing /etc/init.d/mysqld start
Starting MySQL... [ OK ]
Start of mysql-server complete
[root@virt-cluster02 yum.repos.d]# rg_test test /etc/cluster/cluster.conf stop service mysql-server
Running in test mode.
Stopping mysql-server...
<info> Executing /etc/init.d/mysqld stop
Shutting down MySQL. [ OK ]
<info> Removing IPv4 address 115.144.181.43/26 from eth0
<info> unmounting /MYSQL_DATA
volume_list=["VG01", "@virt-node2"]
<notice> Deactivating VG02/mysql_data
<notice> Making resilient : lvchange -an VG02/mysql_data
/usr/share/cluster/lvm_by_lv.sh: line 46: [: too many arguments
<notice> Resilient command: lvchange -an VG02/mysql_data --config devices{filter=["a|/dev/sdb1|","a|/dev/sdc1|","a|/dev/sdd1|","a|/dev/sde1|","a|/dev/sdf1|","a|/dev/sdg1|","a|/dev/vda8|","r|.*|"]}
Reading all physical volumes. This may take a while...
Found volume group "VG03" using metadata type lvm2
Found volume group "VG02" using metadata type lvm2
Found volume group "VG01" using metadata type lvm2
<notice> Removing ownership tag (virt-node2) from VG02/mysql_data
Logical volume "mysql_data" changed
/usr/share/cluster/lvm_by_lv.sh: line 256: [: ==: unary operator expected
Stop of mysql-server complete
[root@ ]# clusvcadm -e mysql-server
Local machine trying to enable service:mysql-server...Success
service:mysql-server is now running on virt-node2
자 정상적으로 enbled되고 cluster가 정상적으로 작동되는것을 확인할수 있습니다.
[root@ ]# clustat
Cluster Status for virt_RHCS @ Wed Oct 17 17:23:26 2012
Member Status: Quorate
Member Name ID Status
------ ---- ---- ------
virt-node1 1 Online, rgmanager
virt-node2 2 Online, Local, rgmanager
/dev/sda1 0 Online, Quorum Disk
Service Name Owner (Last) State
------- ---- ----- ------ -----
service:mysql-server virt-node2 started
service:tomcat-server virt-node1 started
그렇다 다된줄알았다 그러다 dmesg을 쳤는데 이게 왠일인가 오류메시지가아니던가....
[root@ /]# dmesg
EXT4-fs (dm-4): mounted filesystem with ordered data mode
JBD: barrier-based sync failed on dm-4-8 - disabling barrier
대충 보면 write barrier에 대한 sync도중 failed된 내용인데 어쩌라는건가 동작은 제대로하는데...
그래도 미심쩍다 함 찾아보자
I get "kernel: JBD: barrier-based sync failed on dm-0 - disabling barriers" messages when I use "barrier=1" option on a file system in a logical volume, why?
Red Hat Enterprise Linux 5
ext3 filesystems mounted with "barrier=1" option
ext4 filesystems mounted with "barrier=1" or "default" mount options.
barrier=1 is a default mount option for ext4
그래 알았다 ext4에서는 barrier=1이 defaults란다.....
중요한건 뭐???? 그래....장애냐 아니냐다....그것만 아니면 된다 확실하면 된다는것이다.
해결
This is a harmless error message and can be ignored.
- 신경쓰지말란다.......
The kernel disables barrier and executes the write as if barrier=0 was set.
If the error message is occurring too regularly, disabling barriers for non-compatible devices (such as LVM, mdraid and cache-writeback devices) will prevent the messages from occurring.
- 뭐라고????신경쓰지말라면서.....비호환장치를 통해 규칙적으로 에러메시지가 발생한다면, barrier를 끄라는것인가??!
그럴순 없다....
This will have no effect on writing data, because barriers cannot be used with these devices anyway.
- 그렇다.....결론은 데이타에 손상주지 않으니 신경쓰지말란다. 결론적으로 장애상황으로는 발생하지 않으니 걱정말라는내용이다.ㅠㅠ
끝으로......
특별한 이슈없이 정상적으로 프로세스 및 데이타가 유지되는것을 확인할수 있으며, 시간또한 별로 걸리지 않는 것을 확인할수 있다.
속도에 대해서는 iscsi환경이기 때문에 ext4에 대한 퍼포먼스BMT에 대한 내용은 다른 문서를 참고하시면 될듯하다
'Linux is.... > My Skills' 카테고리의 다른 글
| netstat을 이용한 Socket연결상태확인 (0) | 2013.11.17 |
|---|---|
| Clustered Samba?CTDB?그게 뭘까... (0) | 2013.11.17 |
| ifcfg-ethN파일내에 UUID는 무엇을 뜻하는 것인가? (0) | 2013.11.17 |
| SCSI Addressing에 대한 이해 (0) | 2013.11.17 |
| O/S상에서 물리적인 CPU 갯수 확인해보자 (0) | 2013.11.17 |
