Buffalo NAS-Central Forums

Welcome to the Linkstation Wiki community
It is currently Sun Aug 19, 2018 9:48 pm

All times are UTC+01:00




Post new topic  Reply to topic  [ 67 posts ]  Go to page 1 2 3 4 5 Next
Author Message
 Post subject: XFS fixed?
PostPosted: Mon Mar 17, 2008 8:59 pm 
Offline
Newbie

Joined: Mon Mar 17, 2008 8:57 pm
Posts: 10
I thought this post to the XFS mailing list might be of interest:


Message-ID: <47DB4181.7040603@sandeen.net>
Date: Fri, 14 Mar 2008 22:24:49 -0500
From: Eric Sandeen <sandeen@sandeen.net>
To: xfs-oss <xfs@oss.sgi.com>
CC: patches@arm.linux.org.uk
Subject: [PATCH] fix dir2 shortform structures on ARM old ABI

This should fix the longstanding issues with xfs and old ABI
arm boxes, which lead to various asserts and xfs shutdowns,
and for which an (incorrect) patch has been floating around
for years. (Said patch made ARM internally consistent, but
altered the normal xfs on-disk format such that it looked
corrupted on other architectures):
http://lists.arm.linux.org.uk/lurker/me ... f21a2.html

Old ABI ARM has interesting packing & padding; for example
on ARM old abi:

struct xfs_dir2_sf_entry {
__uint8_t namelen; /* 0 1 */

/* XXX 3 bytes hole, try to pack */

xfs_dir2_sf_off_t offset; /* 4 4 */
__uint8_t name[1]; /* 8 1 */

/* XXX 3 bytes hole, try to pack */

xfs_dir2_inou_t inumber; /* 12 8 */

/* size: 20, cachelines: 1 */
/* sum members: 14, holes: 2, sum holes: 6 */
/* last cacheline: 20 bytes */
};

but on x86:

struct xfs_dir2_sf_entry {
__uint8_t namelen; /* 0 1 */
xfs_dir2_sf_off_t offset; /* 1 2 */
__uint8_t name[1]; /* 3 1 */
xfs_dir2_inou_t inumber; /* 4 8 */

/* size: 12, cachelines: 1 */
/* last cacheline: 12 bytes */
};

... this sort of discrepancy leads to problems.

I've verified this patch by comparing the on-disk structure
layouts using pahole from the dwarves package, as well as
running through a bit of xfsqa under qemu-arm, modified so
that the check/repair phase after each test actually executes
check/repair from the x86 host, on the filesystem populated
by the arm emulator. Thus far it all looks good.

There are 2 other structures with extra padding at the end,
but they don't seem to cause trouble. I suppose they could
be packed as well: xfs_dir2_data_unused_t and xfs_dir2_sf_t.

Note that userspace needs a similar treatment, and any
filesystems which were running with the previous rogue
"fix" will now see corruption (either in the kernel, or
during xfs_repair) with this fix properly in place; it
may be worth teaching xfs_repair to identify and fix that
specific issue.

Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

---

Index: linux-2.6.24/fs/xfs/linux-2.6/xfs_linux.h
===================================================================
--- linux-2.6.24.orig/fs/xfs/linux-2.6/xfs_linux.h
+++ linux-2.6.24/fs/xfs/linux-2.6/xfs_linux.h
@@ -300,4 +300,11 @@ static inline __uint64_t howmany_64(__ui
return x;
}

+/* ARM old ABI has some weird alignment/padding */
+#if defined(__arm__) && !defined(__ARM_EABI__)
+#define __arch_pack __attribute__((packed))
+#else
+#define __arch_pack
+#endif
+
#endif /* __XFS_LINUX__ */
Index: linux-2.6.24/fs/xfs/xfs_dir2_sf.h
===================================================================
--- linux-2.6.24.orig/fs/xfs/xfs_dir2_sf.h
+++ linux-2.6.24/fs/xfs/xfs_dir2_sf.h
@@ -62,7 +62,7 @@ typedef union {
* Normalized offset (in a data block) of the entry, really xfs_dir2_data_off_t.
* Only need 16 bits, this is the byte offset into the single block form.
*/
-typedef struct { __uint8_t i[2]; } xfs_dir2_sf_off_t;
+typedef struct { __uint8_t i[2]; } __arch_pack xfs_dir2_sf_off_t;

/*
* The parent directory has a dedicated field, and the self-pointer must
@@ -76,14 +76,14 @@ typedef struct xfs_dir2_sf_hdr {
__uint8_t count; /* count of entries */
__uint8_t i8count; /* count of 8-byte inode #s */
xfs_dir2_inou_t parent; /* parent dir inode number */
-} xfs_dir2_sf_hdr_t;
+} __arch_pack xfs_dir2_sf_hdr_t;

typedef struct xfs_dir2_sf_entry {
__uint8_t namelen; /* actual name length */
xfs_dir2_sf_off_t offset; /* saved offset */
__uint8_t name[1]; /* name, variable size */
xfs_dir2_inou_t inumber; /* inode number, var. offset */
-} xfs_dir2_sf_entry_t;
+} __arch_pack xfs_dir2_sf_entry_t;

typedef struct xfs_dir2_sf {
xfs_dir2_sf_hdr_t hdr; /* shortform header */


Top
   
 Post subject: Re: XFS fixed?
PostPosted: Mon Mar 17, 2008 9:40 pm 
Offline
Site Admin
User avatar

Joined: Mon Jul 11, 2005 7:19 am
Posts: 7703
Location: Austria, Vienna
very interesting. thx for posting this here. anyone time for testing this?

EDIT: just noticed that nobody would be able to test this as the userspace apps also need fixes. or maybe i am wrong?

_________________
LS1 (2.6 kernel, foonas svn1062, 750 GB, UBoot 1.2) & LS Pro (FreeLink/jtymod/GenLink, changes all the time)
Thx to all donators!


Top
   
 Post subject: Re: XFS fixed?
PostPosted: Mon Mar 17, 2008 9:48 pm 
Offline
Newbie

Joined: Mon Mar 17, 2008 8:57 pm
Posts: 10
My only ARM system is a live system with important files on it. I am probably going to hold off until xfs_repair or another program can fix any padding issues on my existing XFS partition.


Top
   
 Post subject: Re: XFS fixed?
PostPosted: Mon Mar 17, 2008 10:04 pm 
Offline
Site Admin
User avatar

Joined: Mon Jul 11, 2005 7:19 am
Posts: 7703
Location: Austria, Vienna
yeah, after this bug is floating around for a so long time i would expect that xfs_repair will have the functionality to fix it properly.

_________________
LS1 (2.6 kernel, foonas svn1062, 750 GB, UBoot 1.2) & LS Pro (FreeLink/jtymod/GenLink, changes all the time)
Thx to all donators!


Top
   
 Post subject: Re: XFS fixed?
PostPosted: Mon Mar 17, 2008 11:40 pm 
Offline
Site Admin
User avatar

Joined: Tue Jul 12, 2005 11:26 am
Posts: 3701
Location: JAPAN
I will test this out on my system.

Built and rebooted. Formatted spare drive:

Code:
LS-GL7D6:/# mkfs.xfs -f /dev/sdb1
meta-data=/dev/sdb1              isize=256    agcount=4, agsize=2441878 blks
         =                       sectsz=512   attr=2
data     =                       bsize=4096   blocks=9767512, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096
log      =internal log           bsize=4096   blocks=4769, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=0
realtime =none                   extsz=4096   blocks=0, rtextents=0


Just copied my /usr/src directory onto it:

/dev/sdb1 38G 595M 37G 2% /mnt/usbdisk1

Directory looked okay and still could access files. A couple of mounts and un-mounts and more writes were still okay. An xfs_check ran alright. Still needs further checks but looking okay at present but then I did not have too many problems with XFS on this box before until moving to ARMel Debian which I am still running.

UPDATED: lb_worm

Further testing of this has not been good. I had copied 13GB to the drive with no issues. On starting to use the filesystem by moving and copying individual files resulted in metadata corruption messages; no dmesg though. This has resulted in having to do multiple xfs_repair's to the filesystem to continue using it.

_________________
LS used as PVR and streaming source


Top
   
 Post subject: Re: XFS fixed?
PostPosted: Wed Mar 19, 2008 12:24 pm 
Offline
Regular Member

Joined: Sun Apr 01, 2007 7:02 pm
Posts: 148
Location: Spain
I doubt it does any good, since the patch has an effect only for arm oabi, and the linkstation kernel is eabi.


Top
   
 Post subject: Re: XFS fixed?
PostPosted: Wed Mar 19, 2008 12:42 pm 
Offline
Newbie

Joined: Mon Mar 17, 2008 8:57 pm
Posts: 10
luca, I noticed that too. From the XFS list, it seems that they think this shouldn't be an issue on the EABI kernel. lb_worm, would you be willing to follow up with the patch author or the XFS list about your problems given that you are in a position to actually test it? It would be nice to get this problem fixed finally.


Last edited by idegen on Wed Mar 19, 2008 6:49 pm, edited 1 time in total.

Top
   
 Post subject: Re: XFS fixed?
PostPosted: Wed Mar 19, 2008 4:41 pm 
Offline
Regular Member

Joined: Sun Apr 01, 2007 7:02 pm
Posts: 148
Location: Spain
Well, I sent my test image to Eric Sandeen and he tested it on 2.6.26-rc2 on arm-eabi:
Eric Sandeen wrote:
So, made an xfs_metadump image of that (just smaller, easier to move
around):

wget xxxxxxxxxxxxxxxxxxxxxxxxxxxxx

(don't worry, it's not linked from anywhere, I can take it down when you
grab it) and then:

bunzip2 sda2-meta.img.bz2
xfs_mdrestore sda2-meta.img sda2.img
mkdir mnt
mount -o loop sda2.img mnt/
ls mnt
ls mnt/sbin

and this all worked fine for me on 2.6.25, eabi.

Linux fedora-arm 2.6.25-rc2 #2 Sat Feb 23 13:58:22 CST 2008 armv5tejl
armv5tejl armv5tejl GNU/Linux

Can you try the same steps on your box?


I see that the orion.git repository only has rc1, is there a safe enough way to test it with rc2 on the linkstation?


Top
   
 Post subject: Re: XFS fixed?
PostPosted: Wed Mar 19, 2008 6:18 pm 
Offline
Site Admin
User avatar

Joined: Tue Jul 12, 2005 11:26 am
Posts: 3701
Location: JAPAN
I have 2.6.25-rc3 running with EABI and ARMel Debian still have issues with XFS. i will check out what he has suggested and post my findings.

_________________
LS used as PVR and streaming source


Top
   
 Post subject: Re: XFS fixed?
PostPosted: Wed Mar 19, 2008 7:27 pm 
Offline
Total Newbie

Joined: Wed Mar 19, 2008 7:23 pm
Posts: 4
There's another change in 2.6.25-rc4 which fixed an xfs issue, see:

commit 94a3f78566ef98a48814d82892f28bb741624cb8
Author: Lennert Buytenhek <email>
Date: Sat Feb 23 00:23:48 2008 +0100

[ARM] 4837/1: make __get_unaligned_*() return unsigned types

Eric Sandeen tracked an XFS on ARM corruption bug down to a function
under fs/xfs/ involving some get_unaligned() calls on u64 pointers.

so if you test newer kernels I'd suggest getting at least one that new.

Also, FWIW, if you guys are having xfs problems, you should bring details to the xfs list, or the xfs developers will never know, and be unable to help... :)

-Eric


Top
   
 Post subject: Re: XFS fixed?
PostPosted: Wed Mar 19, 2008 7:33 pm 
Offline
Regular Member

Joined: Sun Apr 01, 2007 7:02 pm
Posts: 148
Location: Spain
lb_worm, are you subscribed to the xfs mailing list? Because Eric rightly suggested that if we (actually you, since I'm doing nothing with the kernel) keep the problems here it'll be difficult to have a solution:
Eric Sandeen wrote:
FWIW... I think you guys would have much more luck if you bring these
problems to the xfs list. Keeping them on the forums pretty much
ensures that no xfs developer will ever see them. :)

If you guys provide failure cases, messages, xfs_metadump images, and
kernel details I'm sure we can work through some of these issues.


edit: eric, you beat be for a few minutes while I was composing my message :)


Top
   
 Post subject: Re: XFS fixed?
PostPosted: Wed Mar 19, 2008 9:52 pm 
Offline
Site Admin
User avatar

Joined: Tue Jul 12, 2005 11:26 am
Posts: 3701
Location: JAPAN
No I am not on the XFS mailing list. I will have a look at this. These XFS issues have been on-going and brad had spent some time looking into the problem and surfing the available mailing list for solutions. I will look at this latest patch and see if that addresses anything, thanx.

PS: My wifi is painfully slow at the moment and I can not get onto the site for the patch. Will keep trying :)

UPDATED: lb_worm

Got it and currently rebuilding my kernel.

_________________
LS used as PVR and streaming source


Top
   
 Post subject: Re: XFS fixed?
PostPosted: Wed Mar 19, 2008 10:33 pm 
Offline
Site Admin
User avatar

Joined: Tue Jul 12, 2005 11:26 am
Posts: 3701
Location: JAPAN
No joy. formated drive:

Code:
LS-GL7D6:~# mkfs.xfs -f /dev/sdb1
meta-data=/dev/sdb1              isize=256    agcount=4, agsize=2441878 blks
         =                       sectsz=512   attr=2
data     =                       bsize=4096   blocks=9767512, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096
log      =internal log           bsize=4096   blocks=4769, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=0
realtime =none                   extsz=4096   blocks=0, rtextents=0


and copied data to it. Unmounted and removed. Attached drive and tried re-mount, dmesg showed:

Code:
XFS mounting filesystem sdb1
Starting XFS recovery on filesystem: sdb1 (logdev: internal)
XFS: xlog_recover_process_data: bad flag
XFS: log mount/recovery failed: error 5
XFS: log mount failed


an xfs_repair run:

Code:
LS-GL7D6:~# xfs_repair -L /dev/sdb1
Phase 1 - find and verify superblock...
Phase 2 - using internal log
        - zero log...
ALERT: The filesystem has valuable metadata changes in a log which is being
destroyed because the -L option was used.
        - scan filesystem freespace and inode maps...
        - found root inode chunk
Phase 3 - for each AG...
        - scan and clear agi unlinked lists...
        - process known inodes and perform inode discovery...
        - agno = 0
        - agno = 1
        - agno = 2
        - agno = 3
        - process newly discovered inodes...
Phase 4 - check for duplicate blocks...
        - setting up duplicate extent list...
        - check for inodes claiming duplicate blocks...
        - agno = 0
        - agno = 1
        - agno = 2
        - agno = 3
Phase 5 - rebuild AG headers and trees...
        - reset superblock...
Phase 6 - check inode connectivity...
        - resetting contents of realtime bitmap and summary inodes
        - traversing filesystem ...
        - traversal finished ...
        - moving disconnected inodes to lost+found ...
Phase 7 - verify and correct link counts...
done


Then re-mounted okay. Filesystem at present intact.

_________________
LS used as PVR and streaming source


Top
   
 Post subject: Re: XFS fixed?
PostPosted: Wed Mar 19, 2008 11:40 pm 
Offline
Total Newbie

Joined: Wed Mar 19, 2008 7:23 pm
Posts: 4
Well, that looks like quite a different problem than others that were reported before, no? Previously there were reports of corruption and shutdown when the fs was in use?

When you say "Unmounted and removed. Attached drive and tried re-mount" was this all on the arm box, and was this old or new ABI?

If you can bring this problem over to the xfs mailing list I'll help you work through it. You might instrument xlog_recover_process_data() to see what the bad flag value was, or we might just dump out the problematic log for inspection....


Top
   
 Post subject: Re: XFS fixed?
PostPosted: Thu Mar 20, 2008 9:21 am 
Offline
Site Admin
User avatar

Joined: Tue Jul 12, 2005 11:26 am
Posts: 3701
Location: JAPAN
The was all on the ARM box using the new ABI (EABI kernel and Debian). Can you ping me a link to the XFS mailing list, thanx.

_________________
LS used as PVR and streaming source


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic  [ 67 posts ]  Go to page 1 2 3 4 5 Next

All times are UTC+01:00


Who is online

Users browsing this forum: No registered users and 5 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Limited