《Kubernetes集群跨节点挂载CephFS》的评论

作者：bigwhite

bigwhite — Mon, 11 Feb 2019 03:03:13 +0000

没试过。不过建议参考一下：http://docs.ceph.com/docs/mimic/cephfs/quota 自己探索一下:)。不过cephfs的quota可能对操作系统内核版本、k8s版本有依赖。这篇文章有描述：https://www.cnblogs.com/ltxdzh/p/9173706.html 也可以参考一下。

作者：li

li — Mon, 11 Feb 2019 02:00:39 +0000

请教下,cephfs如果使用k8s挂载的话如何限制挂载指定目录的大小呢?

作者：bigwhite

bigwhite — Wed, 18 Jul 2018 03:34:24 +0000

“EXT4-fs (rbd0): VFS: Can’t find ext4 filesystem” , 你的各个物理节点上使用文件系统是ext4么？

作者：jiu

jiu — Wed, 18 Jul 2018 03:03:08 +0000

是，每个节点都有
dmesg

[232683.458167] EXT4-fs (rbd0): VFS: Can’t find ext4 filesystem
[232684.472345] EXT4-fs (rbd0): mounted filesystem with ordered data mode. Opts: (null)
[232688.634998] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[232688.667143] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[233056.592089] libceph: mon0 192.168.181.93:6789 session established
[233056.594443] libceph: client25404 fsid 20631ce0-4c16-4b5e-9f45-759943951a45
[233056.648198] rbd: rbd0: capacity 21474836480 features 0×1
[233057.064804] EXT4-fs (rbd0): mounted filesystem with ordered data mode. Opts: (null)
[233059.633525] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[233059.633993] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[234213.872978] systemd-journald[487]: Failed to set file attributes: Operation not supported
[234213.919069] systemd-journald[487]: Failed to set file attributes: Operation not supported
[236614.545464] systemd-journald[487]: Failed to set file attributes: Operation not supported

作者：jiu

jiu — Wed, 18 Jul 2018 02:52:46 +0000

dmesg

作者：bigwhite

bigwhite — Wed, 18 Jul 2018 02:33:21 +0000

你的每个k8s节点上都存在/etc/ceph/admin.secret文件? 另外出错时，dmesg | tail的输出是啥?

作者：jiu

jiu — Wed, 18 Jul 2018 01:57:57 +0000

没有看明白啊，他说是由于 encoding的问题，但是我的.secret 文件就是utf8的，里边的内容是base64的。

作者：bigwhite

bigwhite — Wed, 18 Jul 2018 00:43:31 +0000

我没遇到过类似问题。不过k8s上的issue中有与你遇到的问题类似的issue，请参考这个解决：https://github.com/kubernetes/kubernetes/issues/39227

作者：jiu

jiu — Tue, 17 Jul 2018 11:27:05 +0000

这是我的pod,请帮忙查看一下问题，谢谢

apiVersion: v1
kind: Pod
metadata:
name: ceph-pod2-with-secret
spec:
containers:
– name: ceph-ubuntu2
image: ubuntu:14.04
command: ["tail", "-f", "/var/log/bootstrap.log"]
volumeMounts:
– name: ceph-vol2
mountPath: /mnt/cephfs/data
readOnly: false
volumes:
– name: ceph-vol2
cephfs:
monitors:
– 192.168.181.93:6789
user: admin
secretFile: “/data/cephfs/admin.secret”
readOnly: false
~

作者：jiu

jiu — Tue, 17 Jul 2018 11:25:26 +0000

你好，问一下我本地测试可以通过，但是Pod测试的时候报错。
df -h
192.168.181.93:6789:/ 95G 0 95G 0% /mnt

kubectl describe pod ceph-pod2-with-secret

Mounting arguments: -t ceph -o name=admin,secretfile=/data/cephfs/admin.secret 192.168.181.93:6789:/ /var/lib/kubelet/pods/2af20094-89b3-11e8-a44a-000c29cc5e76/volumes/kubernetes.io~cephfs/ceph-vol2
Output: mount: wrong fs type, bad option, bad superblock on 192.168.181.93:6789:/,
missing codepage or helper program, or other error

In some cases useful info is found in syslog – try
dmesg | tail or so.

问一下，这是什么原因呢？