2019年7月16日 星期二

pandas note

pandas note

pandas note

https://pandas.pydata.org

read_csv

import pandas as pd
pd.read_csv('<fileanme>', delimiter='<delimeter>', names=['col1', 'col2'])

unique

data['status'].unique()

return

array(['activated', 'running', 'closed', 'transferring', 'finished',
       'holding'], dtype=object)

to_datetime

convert string/int(unix time) to datetime object

pd.to_datetime(data['time'], unit='s')

2019年7月10日 星期三

multipath

multipath

setup multipath

caution
this configuration only work on sl6+promise machine.

  • install packages
    • xfsprogs
    • device-mapper-multipath
  • /etc/multipath.conf
  • reboot machine

/etc/multipath.conf

defaults {
	user_friendly_names yes
}

blacklist {
	devnode "^(sd)[a-z][[0-9]*]"
}

devices {
		device {
		vendor  "Promise"
		product "VTrak"
		path_grouping_policy  group_by_prio
		#getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
		getuid_callout  "/lib/udev/scsi_id --whitelisted --device=/dev/%n"
		#prio_callout "/sbin/mpath_prio_intel /dev/%n"
		prio  alua
		path_checker  tur
		path_selector "round-robin 0"
		hardware_handler  "1 alua"
		failback  immediate
		rr_weight uniform
		rr_min_io 10000
		no_path_retry queue
		features  "1 queue_if_no_path"
		product_blacklist "VTrak V-LUN"
	}
}
  • checking multipath
    if multipath been setup correctly, the output of df -h should be:
Filesystem  Size  Used Avail Use% Mounted on
/dev/sdq2 1.8T  107G  1.6T 7% /
tmpfs  47G 0 47G 0% /dev/shm
/dev/sdq1 194M 82M  103M  45% /boot
/dev/mapper/mpathi 21T  235G 20T 2% /data01
/dev/mapper/mpathb 21T  252G 20T 2% /data02
/dev/mapper/mpatha 21T  243G 20T 2% /data03
/dev/mapper/mpathc 21T  249G 20T 2% /data04
/dev/mapper/mpathg 21T  252G 20T 2% /data05
/dev/mapper/mpathd 21T  255G 20T 2% /data06
/dev/mapper/mpathe 21T  238G 20T 2% /data07
/dev/mapper/mpathh 13T  160G 13T 2% /data08

lsblk

NAME            MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINT
sda               8:0    0   20T  0 disk  
└─mpathi (dm-0) 253:0    0   20T  0 mpath /data01
sdb               8:16   0   20T  0 disk  
└─mpathb (dm-1) 253:1    0   20T  0 mpath /data02
sdd               8:48   0   20T  0 disk  
└─mpatha (dm-2) 253:2    0   20T  0 mpath /data03
sdc               8:32   0   20T  0 disk  
└─mpathc (dm-3) 253:3    0   20T  0 mpath /data04
sde               8:64   0   20T  0 disk  
└─mpathg (dm-4) 253:4    0   20T  0 mpath /data05
sdf               8:80   0   20T  0 disk  
└─mpathd (dm-5) 253:5    0   20T  0 mpath /data06
sdg               8:96   0   20T  0 disk  
└─mpathe (dm-6) 253:6    0   20T  0 mpath /data07
sdh               8:112  0 12.8T  0 disk  
└─mpathh (dm-7) 253:7    0 12.8T  0 mpath /data08
sdq              65:0    0  1.8T  0 disk  
├─sdq1           65:1    0  200M  0 part  /boot
├─sdq2           65:2    0  1.8T  0 part  /
└─sdq3           65:3    0    8G  0 part  [SWAP]
sdi               8:128  0   20T  0 disk  
└─mpathi (dm-0) 253:0    0   20T  0 mpath /data01
sdj               8:144  0   20T  0 disk  
└─mpathb (dm-1) 253:1    0   20T  0 mpath /data02
sdl               8:176  0   20T  0 disk  
└─mpatha (dm-2) 253:2    0   20T  0 mpath /data03
sdk               8:160  0   20T  0 disk  
└─mpathc (dm-3) 253:3    0   20T  0 mpath /data04
sdm               8:192  0   20T  0 disk  
└─mpathg (dm-4) 253:4    0   20T  0 mpath /data05
sdo               8:224  0   20T  0 disk  
└─mpathe (dm-6) 253:6    0   20T  0 mpath /data07
sdn               8:208  0   20T  0 disk  
└─mpathd (dm-5) 253:5    0   20T  0 mpath /data06
sdp               8:240  0 12.8T  0 disk  
└─mpathh (dm-7) 253:7    0 12.8T  0 mpath /data08

blkid

/dev/sdq1: UUID="bf6357f7-44e8-4f64-9a10-0a58969f070f" TYPE="ext3" 
/dev/sdq2: UUID="77bddbd6-8e9a-4712-8ad4-5dd73ac2538b" TYPE="ext4" 
/dev/sdq3: UUID="e66ef8f7-6c99-44fe-9802-62e7dd8d357f" TYPE="swap" 
/dev/sdb: LABEL="data02" UUID="c88f14b9-056f-443a-ba91-3b842495501e" TYPE="xfs" 
/dev/sdd: LABEL="data03" UUID="cce1537b-31dc-4f5e-b34f-084cefa31604" TYPE="xfs" 
/dev/sdf: LABEL="data06" UUID="a51935d3-56e6-4f42-b302-941d8f0480b1" TYPE="xfs" 
/dev/sdh: LABEL="data08" UUID="e9c3ce71-823d-4121-a1ee-f18150e61e96" TYPE="xfs" 
/dev/sdi: LABEL="data01" UUID="38c17b01-23d7-4e40-b2c4-70a0fe72ec70" TYPE="xfs" 
/dev/sdk: LABEL="data04" UUID="07518c1e-5ff3-44b1-9937-79c446644401" TYPE="xfs" 
/dev/sdm: LABEL="data05" UUID="245602d8-e42e-467c-b81b-4efcb9e659aa" TYPE="xfs" 
/dev/sdo: LABEL="data07" UUID="27208abf-7b8c-4e00-afc2-c1675174a9c2" TYPE="xfs" 
/dev/mapper/mpathi: LABEL="data01" UUID="38c17b01-23d7-4e40-b2c4-70a0fe72ec70" TYPE="xfs" 
/dev/mapper/mpathb: LABEL="data02" UUID="c88f14b9-056f-443a-ba91-3b842495501e" TYPE="xfs" 
/dev/mapper/mpatha: LABEL="data03" UUID="cce1537b-31dc-4f5e-b34f-084cefa31604" TYPE="xfs" 
/dev/mapper/mpathc: LABEL="data04" UUID="07518c1e-5ff3-44b1-9937-79c446644401" TYPE="xfs" 
/dev/mapper/mpathg: LABEL="data05" UUID="245602d8-e42e-467c-b81b-4efcb9e659aa" TYPE="xfs" 
/dev/mapper/mpathd: LABEL="data06" UUID="a51935d3-56e6-4f42-b302-941d8f0480b1" TYPE="xfs" 
/dev/mapper/mpathe: LABEL="data07" UUID="27208abf-7b8c-4e00-afc2-c1675174a9c2" TYPE="xfs" 
/dev/mapper/mpathh: LABEL="data08" UUID="e9c3ce71-823d-4121-a1ee-f18150e61e96" TYPE="xfs"

2019年7月8日 星期一

4/7

Y

4/7和Y以及Y的同學去爬山,現在想想那應該是喜歡上爬山的契機,故此留個記錄。

走的路線是無耳茶壺山=>半屏山=>燦光寮山=>黃金神社,不難,但很熱。過程中有拉繩,有泥巴地,相當有趣。

下了燦光寮山後再往前走一些,向右望去,沿路走過的山在眼前如畫卷般展開,真的好漂亮。好難形容,那不僅是風景優美造成的漂亮,那是親身走過,才能發現的漂亮。