I had a lost+found folder in all Unix file systems I used since the 80s. It's where fsck places files that it found during a scan and can't figure out to which directory they belong. Sometimes I found stuff in there.
From what I googled XFS, Btrfs and ZFS don't use lost+found. It's a thing of the old not journaled filesystems and of the ext family.
XFS does use /lost+found, it calls it the "orphanage directory" and xfs_repair reparents children of corrupt directories there.
Based on comments in the kernel source, it seems like the userspace fsck for JFS and F2FS will also sometimes create /lost+found. There might be more that do.
XFS filesystems do not have a "/lost+found" directory in their normal state.
In the very rare occasions when one has to run "xfs_repair", it will create a "/lost+found" directory, if it is required for recovered files.
After the repair and after investigating whether the recovered files contain useful data or not (and after moving the useful files elsewhere), one should normally delete the "/lost+found" directory, because it is no longer needed.
XFS as implemented in RHEL8+ (the only places i've used it in anger) tends to handle being full very badly, leading to system lockups and blocked tasks necessitating a hard reboot. Worse yet is when it's in this state the journal fills and nothing can be done with the volume.
To recover from this on a volume mounted at boot mandates going to either a live disk, or stopping boot in initramfs and running xfs_repair there, I've fruitlessly attempted to play back the journal on many separate occasions by attempting to mount the filesystem (again causing a lock up due to no space) in that state you have to drop the journal, run xfs_repair and then clean up the detritus from /lost+found (and then the location that caused the disk to fill altogether).
EXT4 has other issues certainly, but at least it reserves blocks for the root user explicitly so the system doesn't stop.
Don't know if this is a regression from before, since in the RHEL 5/6 days I used XFS filesystems as my default filesystem on large storage-pools. Since XFS doesn't have a shrink option, I would create filesystems of a few gigabytes in size, and grow them whenever needed. I mostly used them for monthly archiving of uploads to a customer's website, so there would be a YYYY-MM lvm volume with an XFS filesystem, and during the month it would be grown automatically from a cronjob if space got tight.
I'm quite sure I must have had a bunch of full filesystems there, and never ran into any crashing issues with full XFS filesystems (though these were not the 'root' filesystem). But even on my current laptop (with debian 12/13) I'm running XFS on all filesystems (besides /boot and /efi), and they report being full often enough without any crashes/reboots.
The last time when I have used RHEL was decades ago, so I do not have any idea about what happens there.
On the other hand, I have been using XFS since 2005 (since when I have transitioned from 32-bit Linux to 64-bit Linux), on a great variety of hardware systems, servers, desktops, mini-PCs, laptops.
My file systems are typically mostly full and from time to time I had incidents when some job failed by filling completely the file system and no longer having any space left for writing the remainder of the files being written.
Filling completely the HDD or SSD has never caused any problems. I have always just deleted some files or moved some files to other file systems, and I have continued working. Sometimes I had some downloading in progress, which was halted by the browser because of full disk, and in such cases, after making space, I just resumed the download in the browser.
So I am puzzled by your experience, but I am not very surprised because in Linux there are many obscure configuration options, so the behavior can vary a lot between distributions (I typically use Gentoo). Perhaps your problems were caused by certain daemons that were continuing to make write attempts in the background, which I do not have.
The only problems that I have ever encountered in XFS happened only in early XFS, i.e. 2 decades ago, which was extremely sensitive to power failures, despite being a journaled filesystem. In early XFS, after a power failure, some previously open files were erased, even if they had been open only for reading. Because of this, a power failure frequently bricked the system, by erasing "/etc/fstab".
However, this stupid XFS feature has been corrected many years ago and nowadays power failures normally do not have any effect on XFS, i.e. xfs_repair is normally not needed, even after power failures. That was a bug at the conceptual level, not at the programming level, because the erasure of some files in early XFS was intentional, because it wrongly concluded that they might have been corrupted.
While early XFS was notorious for its fragility against power failures, at that time none of the competing file systems was significantly better, all were buggy. Around the same time, more than two decades ago, I have seen a lot of other filesystems corrupted by power failures, regardless whether they were Windows NTFS or Linux EXT3 or JFS, despite the fact that all were advertised as being resistant to power failures by being journaled. At that time, only one filesystem was completely impervious to power failures, and it was non-journaled, the FreeBSD UFS with "soft updates" (i.e. with a careful ordering of the disk writes, to maintain a consistent state across power failures).
If you truncate a file it doesn't update metadata. That's how you can get back space for the journal log to start cleaning up crap without rebooting the box and/or taking services down.
Even with journaling, you might need one. ZeroFS [0] almost had a lost+found directory (even with the WAL enabled), because you might have consistency issues between your in-memory state and what was flushed, and especially in what order.
ZeroFS ended up not needing recovery at all through atomic, strictly ordered commits [1], but it was far from trivial (and not just a matter of requiring a WAL).
I have a book on my bookshelf, Eric Foxley's Unix for Super-Users. It was published in 1985, and it answers this question on page 52, the first page listed for the entry 'lost+found' in its index.
This is surely not the earliest book mention, is it? (It'll be in earlier man pages, of course.) Google Books does not give me an earlier one, although it does yield another 1985 book.
Fun fact: Foxley cautioned that lost+found must be pre-sized ahead of time, because the fsck of the time did not grow the directory to fit found files.
Foxley gives the manual procedure for sizing the lost+found directory on the aforementioned page 52.
I have the 1986 edition of Fielder's and Hunter's UNIX System Administration and it does not mention any such command in its discussion of lost+found in chapter 3. It references AT&T Unix System 5 Release 2 (or 'UNIX 5.2' as the book puts it).
But Google Books tells me that their later 1991 update, referencing Release 4 and with an updated title to match, does indeed mention mklost+found. So that looks like something that appeared in Release 3 or 4.
I used to port Unix for a living (V6 thru SysVr3) I know it was in Vr2, and I have a man page that says "(last mod. 1/15/87)" which would likely make it SysVr1 or earlier
Foxleys book does not actually claim reason why it need to be preallocated. I've got the book also. Index mentions fsck only page 52, where it reads:
"52 UNIX FOR SUPER-USERS
...
The filestore consistency check is performed by the command fsck (usu-
ally stored as /etc/fsck, but sometimes in /bin), which should be used to check
all discs used as file-systems. It defaults to the list of filestore devices given
in the file /etc/checklist. At this stage, most of the file-systems will not be
mounted, so will be inactive; only the root file-system will be active. The
fsck command goes through each one in turn, reports any inconsistencies in
them, and offers to correct them. The reply to each query is either 'y' for yes
(correct the inconsistency), or 'n' for no (leave the file-system inconsistent).
A parameter '-y' to the command assumes 'yes' replies to all questions, so
that no further interaction is necessary; a parameter '-n' similarly assumes
all 'no' answers, and therefore needs no write permission to the device. Any
'yes' reply may involve the loss of information, such as the complete removal
of a suspect file. Suspect files on the file-system being checked are written to
a directory lost + found on the device if such a directory exists; this directory
must have been created, and be sufficiently large already to hold the names
of all the files involved. This can be ensured by first creating the directory,
then creating a number of files in the directory, and then removing them.
The corrected systems will be consistent, and can later be mounted as and
when required. It may be possible to recover information from deleted files
by looking at the lost + found directory. There should be a lost + found direc-
tory at the head of each mountable file-system.
When checking the root file-system, there are complications, in that it
will be active (even though, because it is root, it will not be formally mounted
as such, but is implicitly mounted as root during the booting process). If
modifications are necessary, they should be completed, and the machine
rebooted without first performing a sync (see section 4.5 below for the nor-
mal procedure for taking a system down). This is to ensure that the disc as
modified by fsck is not overwritten by any in-core information, which may
have been generated from information read from the original corrupt (incon-
sistent) version.
...
"
But I've also read more detailed explanation that recollect is that unless blocks were preallocated, there was a possibility that lost+found need first allocate more blocks directory it already had, it would possibly led to losing some data that would otherwise been able to recover once fsck had advanced further from that point.
Old time UNIX systems directories were just another structured file, which a 'd' bit (like others you changed with chmod) on them, where each record was 16 bytes, which 2 first were the inode number followed by 14 bytes reserved for filename. IIRC linux also had first same limit first filesystems. You could read directory with any program, common feat was to check any odd stuff that "ls" would not show with could hexdump or "od" with some flags and print 16 bytes lines per row. That way you also could see any deleted or moved files from that directory, because directory entry was not quite long otherwise cleared but just clearing that inode reference two bytes.
A Quick look from other books that I have close me now Maurice J. Bach, The Design of Operating UNIX System (-86) contains much more about fsck and filesystem fixing issues scattered few pages in the book and what methods were used to mitigate loss of data. S.R Bourne The Unix System, no mention of fsck at all, at least by looking book automatically generated index.
It could have been some other quite old book I did read, but did not own or anything since BSD4.3-tahoe documentation I've read over the years. But sure it would be nice to read that exact reasoning again from credible sources.
edit: Oh, and you could preallocate also just by adding entries or copying some data to lost+found enough, and then remove entries. Unix traditionally have not compacted and resized directories. They only grow and can be if have been very large slow to traverse. The way to compact is creating another, moving existing data there and then swapping directories.
OK, I found bit more from UNIX System Administration Handbook Third Edition (Evi Nemeth, Garth Snyder, Scott Seebass, Trent R. Hein, with even more authors mentioned) 2001 book following:
"134 UNIX System Administration Handbook
...
The lost+found directory is automatically created when you build a filesystem. It is
used by fsck in emergencies; do not delete it. The lost+found directory has some
extra space prealocated so that fsck can store "unlinked" files there without having
to allocate additional directory entries on an unstable filesystem. Some systems pro-
vide mklost+found command that can recreate this special directory if it is aci-
dentally deleted.
...
"
The occassional "Drive has not been checked in <n> days, forcing check" message on bootup got annoying sometimes, yeah. It could easily take tens of minutes to finish, exactly when I wanted to use the computer!
(At least this is what my memory is telling me. I could be mistaken, but that's what I remember.)
I used to develop SSD firmware and one of things I worked on is making it robust to power failure. The power supplies have lots of capacitance so the voltage drop was slow so we would use a special test board that would disconnect from power and discharge fast to test it.
When you have dirty writes in the kernel that have not yet been written to disk, in the old days of ext2 (before XFS was ported to Linux) if the power would go out, or you would have a bad disk, when fsck.ext2 would run, if files could not be matched to a directory, they would placed in the /lost+found as, and hopefully my memory is intact, as inode numbers, so you would have 1232342343, 123246564 etc and then you would have to look at each file to figure out what it was and where to move it if it was salvageable.
That's what the answers are missing, of course. In some filesystem formats, it's possible either to recover completely from a journal/intent log, or at least to recover everything to the point that recovered files can be placed into the correct directory.
Same here. And I had some pretty f**ed up file systems.
At one point, I had one where the directory structure was completely broken and had circles in it (broken SSD). To be fair, in that particular case, I did not look for lost+found and just wrote a tool to extract the data manually that I was looking for.
The lost+found folder saved our backs once way back when. We had a 1TB NAS running Linux, an enormous amount of storage back then. It was shut down unexpectedly and disorderly and ran a multi hour fsck upon reboot. The volume must have had a shot root inode, as after the machine booted and mounted the volume it was empty. All directories were luckily under lost+found with all their contents.
We once deleted the lost+found folder on an old Unix system* by accident. Things went very badly the next time the system rebooted, fsck did not handle it at all well.
On the UFS and suchlike filesystems, at the point that fsck is rescuing orphaned i-nodes, it still has not fully gone through the process of checking and correcting free list information, or indeed fully eliminating errors from the i-node table. Creating a directory involves allocating a new i-node from an unused slot, and free blocks off the free block list.
Ironically, because they are slightly or grossly different to Unix filesystem formats, on HPFS and FAT this is less of an issue. (FAT usually has unused slots in the root directory that it is sane to use at that point, for example.) CHKDSK on OS/2 did create its \FOUND.nnn files on the fly.
As the submission explains, the lost+found folder has pre-allocated space for the directory entries. From the mklost+found man page [0]:
mklost+found pre-allocates disk blocks to the lost+found directory
so that when e2fsck(8) is being run to recover a file system, it
does not need to allocate blocks in the file system to store a
large number of unlinked files. This ensures that e2fsck will not
have to allocate data blocks in the file system during recovery.
Pre-allocating space without making the directory visible would require more arcane file system magic.
You greatly overestimate the capabilities of computers of the era that this originated in. Major limitations of disk space, speed and addressability combined with limited CPU power and RAM mean that your hypothetical, “it could have been done” would require a significant investment in development and runtime resources all to avoid having a directory in root that would be empty until after a post-crash recovery.
Also remember that these systems would have all been multi-user time-sharing systems, not desktop computers.
they offer private instances to school too, where moderation is left to school policy, and mine seems to be good enough to use whenever i have frontend questions (i'm 10 year into my career and still use my juniors to answer my frontend questions, i think i won't ever change)
lost+found is still used on OpenBSD, seems it is created when needed. Only /home has that directory on my system. IIRC, it was created when a kernel panic happened a few releases ago. Plus some files were placed in it when fsck executed on /home
Same, it's only in /home on my system also. Also /home is pretty much the only directory where I see fsck needing to do a lot of recovery after a power failure. Makes sense I guess, because that's where processes such as web browsers are likely to have lots of files open in RW mode.
Back in the day I accidentally deleted all my stuff because I had it all in a special dir of this user in suse Linux. When I deleted the user, yast deleted everything.
Fortunately I was using ReiserFS at the time and something about its murderous tree data structure made it trivial to undelete.
Reiser_fsck found ALL my stuff, mostly with full dir tree structure in tact and put it all in lost+found
I've last seen those in older Cisco ASA firewall's up to OS version 8.something which had internal or external CompactFlash with VFAT16 filesystems. Usually caused by end of useful life of that CF or if already replaced by someone who did not quite understand device requirements and what it supports.
Either because did not care or understand 32bit VFAT while it works for while, then when CF usage gets over VFAT16 supported 2GB fs gets corrupted, system fails after a while and then you got plenty of those FOUND.xxx files root of that CF drive after boot ran fsck. Those old ASA's did accept and work with 4GB CF's which were available much longer than 2GB versions, but you needed to make max 2GB VFAT16 primary partition and then format it with Linux mkfs.vfat with flags that made sure it's only 16bit version. Once that was done, you could copy files from old CF using something that copies also hidden files and directories too, which there were few there.
ASA used to be some Cisco proprietary OS and was just rebranded PIX firewall, then from 8 oddly hacked linux with grub boot.
Not much of my favourite box as a firewall, but Remote-access VPN features did work quite well quite long and when clustered it was easy to run upgrades each node at time without DTLS and IPsec clients even noticing it.
You can even tell which operating systems xe has used. Both of those are very specific. The \FOUND.nnn directories are something that IBM/Microsoft introduced when they re-did CHKDSK for OS/2, which was what Windows NT inherited. FILEnnnn.CHK in the root directory was the old Microsoft CHKDSK, and also Symantec/Norton SCANDISK, for MS/PC/DR-DOS.
Not really, as it's only once per file system mount, whereas those Windows and MacOS files are sprinkled in most directories with images and almost every non-network drive directory respectively.
How do questions like this make it to the top? It is an obvious thing if you search for it or ask AI, but people seem to just ignore those in favor of generating new human responses.
Thing is, any time I try to replicate something like that, I basically get a flippant response saying to go look elsewhere.
From what I googled XFS, Btrfs and ZFS don't use lost+found. It's a thing of the old not journaled filesystems and of the ext family.
reply