Discussion:
preallocated file size
Terry McCoy
2003-12-02 09:04:06 UTC
Permalink
I have noticed the following behavior when copying a new
large file (1.9 GB) from a NT 4.0 client to a share on a
Samba server. While the copy is in progress an listing of
the destination directory on the Samba server shows that the
file is in the directory with the allocated size of 1.9GB.

It would appear that perhaps the file is being created and
a seek ahead is being done, then the process of coping the
file over from the client to the server is begun. Hence a
method to preallocate disk space for the new file.

My question(s)

Why is this being done?

Is there an configuration option to turn this behavior off?

If it can't be turned off what location(s) in the source is this
behavior being implemented.


The real problem I am having is that, I believe this behavior as
observed on shares that are local file systems on the Samba server
are the cause of problems I am having when the share is not a local
file system on the Samba server but an distributed network file
system.


Other information:

Samba server version 2.0.5a, on a Sun E2 running Solaris 2.6



--
Terry McCoy email: ***@nd.edu
Sr Systems Engineer phone: (219) 631-4274
Enterprise Systems Software
Office of Information Technologies
University of Notre Dame
David Collier-Brown
2003-12-02 09:04:06 UTC
Permalink
Terry McCoy wrote:
[...] t he file is being created and
Post by Terry McCoy
a seek ahead is being done, then the process of coping the
file over from the client to the server is begun. Hence a
method to preallocate disk space for the new file.
Why is this being done?
Is there an configuration option to turn this behavior off?
If it can't be turned off what location(s) in the source is this
behavior being implemented.
It appears to be client-side behavior: it doesn't
happen with smbclient, so I'll assume it's an idiom
used on Windows filesystems to get an early failure,
thus saving the user a long wait just to find out
that the copy failed...

Can you tell us what the result is on the Windows side?
I fear it's: "the program appeared to succeed, but the file
is silently truncated if there isn't enough disk space".
That would be bad (;-)). If it is, can you tell us what
programs suffer from it?

Possible workaround: controlled by an option, set a
per-fd flag on open saying "check first seek-write"
In write, if the flag is set, clear it and check
if the write is to the end of the file. If so, use
statvfs (on Solaris, in this case) to see if the
size will exceed the non-root space left on the disk.

Needless to say, I'd prefer not to do that... it's
a lot of server-side monkey-motion to provide what
**should** be a nicety!

--dave
--
David Collier-Brown, | Always do right. This will gratify some people
185 Ellerslie Ave., | and astonish the rest. -- Mark Twain
Willowdale, Ontario | //www.oreilly.com/catalog/samba/author.html
Work: (905) 415-2849 Home: (416) 223-8968 Email: ***@canada.sun.com
Terry McCoy
2003-12-02 09:04:07 UTC
Permalink
Post by David Collier-Brown
[...] t he file is being created and
Post by Terry McCoy
a seek ahead is being done, then the process of coping the
file over from the client to the server is begun. Hence a
method to preallocate disk space for the new file.
Why is this being done?
Is there an configuration option to turn this behavior off?
If it can't be turned off what location(s) in the source is this
behavior being implemented.
It appears to be client-side behavior: it doesn't
happen with smbclient, so I'll assume it's an idiom
used on Windows filesystems to get an early failure,
thus saving the user a long wait just to find out
that the copy failed...
Yes you are correct, I verified this by having an NT client mount
a share on an NT file server and copy a 160MB file. Sure enough
on the NT server the file was created and its size set to 160MB
while the copy was in progress.

Hmmm... I wonder how they handle incomplete file transfers, but
that's a potential problem for another day :>)
Post by David Collier-Brown
Can you tell us what the result is on the Windows side?
I fear it's: "the program appeared to succeed, but the file
is silently truncated if there isn't enough disk space".
That would be bad (;-)). If it is, can you tell us what
programs suffer from it?
I am still looking into this, the users are using the GUI file manager
and I am not getting precise enough information about what the error
messages are.

Wondering if this is specific to NT or does Win 95/98 behave the same?


Do you know if this a client side option for NT/98/95?




--
Terry McCoy email: ***@nd.edu
Sr Systems Engineer phone: (219) 631-4274
Enterprise Systems Software
Office of Information Technologies
University of Notre Dame
David Collier-Brown
2003-12-02 09:04:07 UTC
Permalink
Post by Terry McCoy
Hmmm... I wonder how they handle incomplete file transfers, but
that's a potential problem for another day :>)
I suspect they'll fail on the write to the intended end of
file, allowing the copy program (in this case file manager,
~= NT explorer) to stop quickly. I fear that subsequent
writes aren't checked...
Post by Terry McCoy
Wondering if this is specific to NT or does Win 95/98 behave the same?
Do you know if this a client side option for NT/98/95?
No, but I can find out on the weekend about 95.

--dave
--
David Collier-Brown, | Always do right. This will gratify some people
185 Ellerslie Ave., | and astonish the rest. -- Mark Twain
Willowdale, Ontario | //www.oreilly.com/catalog/samba/author.html
Work: (905) 415-2849 Home: (416) 223-8968 Email: ***@canada.sun.com
David Collier-Brown
2003-12-02 09:04:07 UTC
Permalink
Post by David Collier-Brown
Post by Terry McCoy
Hmmm... I wonder how they handle incomplete file transfers, but
that's a potential problem for another day :>)
Wondering if this is specific to NT or does Win 95/98 behave the same?
Do you know if this a client side option for NT/98/95?
No, but I can find out on the weekend about 95.
I just did the obvious test, and copying on Win95
appears to start at the beginning, copy for a
considerable number of packets, run out of space
on the server, report it and clean up quietly.

The "optimization" appears to be specific to NT.
--dave
--
David Collier-Brown, | Always do right. This will gratify
185 Ellerslie Ave., | some people and astonish the rest.
Willowdale, Ontario, | -- Mark Twain
CANADA. 416-223-8968 | ***@canada.sun.com
Maulik Desai
2003-12-02 09:04:07 UTC
Permalink
Hi,
Post by David Collier-Brown
Post by Terry McCoy
Hmmm... I wonder how they handle incomplete file transfers, but
that's a potential problem for another day :>)
I suspect they'll fail on the write to the intended end of
file, allowing the copy program (in this case file manager,
~= NT explorer) to stop quickly. I fear that subsequent
writes aren't checked...
A while back I had observed that Samba didn't seem to handle incomplete
file transfers properly. When I used explorer to write a file on samba
server (drag & drop) and canceled it before it was completely written, the
(corrupt) file still existed in the directory instead of being deleted. It
appears that NT handles this correctly by deleting the file if it
was canceled before completely written.

Can someone please verify this? if this is true, then this is a bug
in Samba - right?

I'm using Samba 2.0.5a on RH Linux 6.0; client was NT 4.0 w/s.

thanks,
Maulik
Richard Sharpe
2003-12-02 09:04:07 UTC
Permalink
Hi,
Post by Maulik Desai
Hi,
Post by David Collier-Brown
Post by Terry McCoy
Hmmm... I wonder how they handle incomplete file transfers, but
that's a potential problem for another day :>)
I suspect they'll fail on the write to the intended end of
file, allowing the copy program (in this case file manager,
~= NT explorer) to stop quickly. I fear that subsequent
writes aren't checked...
A while back I had observed that Samba didn't seem to handle incomplete
file transfers properly. When I used explorer to write a file on samba
server (drag & drop) and canceled it before it was completely written, the
(corrupt) file still existed in the directory instead of being deleted. It
appears that NT handles this correctly by deleting the file if it
was canceled before completely written.
Ummm, what do you mean by not completely written. In particular, what
operations did the client use to implement drag and drop?

If NT is deleting a file because the client stoppend sending SMBwrite
requests and aborted the connection, I would be very surprised and unhappy.

If you have a trace, we could get a better idea of what is going on.

I suspect that the client is deleting the file.
Post by Maulik Desai
Can someone please verify this? if this is true, then this is a bug
in Samba - right?
I'm using Samba 2.0.5a on RH Linux 6.0; client was NT 4.0 w/s.
thanks,
Maulik
Regards
-------
Richard Sharpe, ***@ns.aus.com, Master Linux Administrator :-),
Samba (Team member, www.samba.org), Ethereal (Team member, www.zing.org)
Co-author, SAMS Teach Yourself Samba in 24 Hours
Author: First Australian 5-day, intensive, hands-on Linux SysAdmin course
David Collier-Brown
2003-12-02 09:04:08 UTC
Permalink
Post by Richard Sharpe
I suspect that the client is deleting the file.
[after the copy fails]

It does on win95: someone else will have to look
at NT.

--dave
--
David Collier-Brown, | Always do right. This will gratify some people
185 Ellerslie Ave., | and astonish the rest. -- Mark Twain
Willowdale, Ontario | //www.oreilly.com/catalog/samba/author.html
Work: (905) 415-2849 Home: (416) 223-8968 Email: ***@canada.sun.com
jeremy at valinux.com ()
2003-12-02 09:04:08 UTC
Permalink
Post by Maulik Desai
A while back I had observed that Samba didn't seem to handle incomplete
file transfers properly. When I used explorer to write a file on samba
server (drag & drop) and canceled it before it was completely written, the
(corrupt) file still existed in the directory instead of being deleted. It
appears that NT handles this correctly by deleting the file if it
was canceled before completely written.
Can someone please verify this? if this is true, then this is a bug
in Samba - right?
I'm using Samba 2.0.5a on RH Linux 6.0; client was NT 4.0 w/s.
Yep - that would be a bug in Samba if NT handles it correctly
and we don't. Can you test this with Samba 2.0.6 to make sure ?

If we have the problem I'd like to look at fixing it before 2.0.7 gets
shipped.

Cheers,

Jeremy Allison,
Samba Team.
David Collier-Brown
2003-12-02 09:04:08 UTC
Permalink
Just for reference, here's a snippit of the log from a win95
machine talking to Samba 2.0.6, at home.

I interpret this as:
openX /fat_file where fat_file is just too big
to fit on the target partition
write 48128 bytes succeeded for a while, then...
write_data: write failure. Error = No space left on device
close file
SMBunlink \fat_file Client cleans up

--dave
--
David Collier-Brown, | Always do right. This will gratify some people
185 Ellerslie Ave., | and astonish the rest. -- Mark Twain
Willowdale, Ontario | //www.oreilly.com/catalog/samba/author.html
Work: (905) 415-2849 Home: (416) 223-8968 Email: ***@canada.sun.com
-------------- next part --------------
An embedded message was scrubbed...
From: David Collier-Brown <***@canada.sun.com>
Subject: win95 normal behavior
Date: Fri, 04 Feb 2000 19:08:11 -0500
Size: 6484
Url: http://lists.samba.org/archive/samba-technical/attachments/20000207/4bfa3c14/attachment.eml
Andreas Abach
2003-12-02 09:04:08 UTC
Permalink
There is still another bug in Samba. When writing a file to share that
is larger than the free space on the disk the file is allocated with
its full length. During the copy process you'll get an error, but the
file will be left on the samba-share and it will be reported with it's
original size. A full report can be read at
www.lkt.uni-erlangen.de/samba-bug/. The same happens when quotas are
used. I send this problem to this list some weeks ago, but it seams to
unimportant to the mass.

Andreas
Post by Richard Sharpe
Hi,
Post by Maulik Desai
Hi,
Post by David Collier-Brown
Post by Terry McCoy
Hmmm... I wonder how they handle incomplete file transfers, but
that's a potential problem for another day :>)
I suspect they'll fail on the write to the intended end of
file, allowing the copy program (in this case file manager,
~= NT explorer) to stop quickly. I fear that subsequent
writes aren't checked...
A while back I had observed that Samba didn't seem to handle incomplete
file transfers properly. When I used explorer to write a file on samba
server (drag & drop) and canceled it before it was completely written, the
(corrupt) file still existed in the directory instead of being deleted. It
appears that NT handles this correctly by deleting the file if it
was canceled before completely written.
Ummm, what do you mean by not completely written. In particular, what
operations did the client use to implement drag and drop?
If NT is deleting a file because the client stoppend sending SMBwrite
requests and aborted the connection, I would be very surprised and unhappy.
If you have a trace, we could get a better idea of what is going on.
I suspect that the client is deleting the file.
Post by Maulik Desai
Can someone please verify this? if this is true, then this is a bug
in Samba - right?
I'm using Samba 2.0.5a on RH Linux 6.0; client was NT 4.0 w/s.
thanks,
Maulik
Regards
-------
Samba (Team member, www.samba.org), Ethereal (Team member, www.zing.org)
Co-author, SAMS Teach Yourself Samba in 24 Hours
Author: First Australian 5-day, intensive, hands-on Linux SysAdmin course
--
Dipl.-Ing. Andreas Abach * Lehrstuhl fuer Kunststofftechnik Uni Erlangen
Tel.: +49-9131-85297-06 * Fax.: -09 * email: ***@LKT.Uni-Erlangen.de
Am Weichselgarten 9 * D-91058 Erlangen * www.keramikspritzgiessen.de
PGPID 0x5672B1F8 F-Print FD50 85C0 7580 5467 65E7 3BE8 58BC 65F4 5672 B1F8
David Collier-Brown
2003-12-02 09:04:08 UTC
Permalink
Post by Andreas Abach
There is still another bug in Samba. When writing a file to share that
is larger than the free space on the disk the file is allocated with
its full length. During the copy process you'll get an error, but the
file will be left on the samba-share and it will be reported with it's
original size. A full report can be read at
www.lkt.uni-erlangen.de/samba-bug/. The same happens when quotas are
used. I send this problem to this list some weeks ago, but it seams to
unimportant to the mass.
It's being discussed, but Jeremy is away and many of the
other chaps are buried in work.

Thanks for the complete writeup: would you be kind
enough to put a log of the connection
up on the web site, so we can compare it with the
log from win95? I'm willing to try to fix it, with
a little help from real experts like Richard.

My initial thoughts go like this:

typedef struct file_fd_struct {
...
BOOL check_for_write_at_end;
} file_fd_struct;

in smbd/open.c open_file()
fds->check_for_write_at_end = True;
...

in <the initial write>
if ( fds->check_for_write_at_end == True
&& requested_position != 0) {
/* Then application was writing the last block */
/* of the file first to see if we have enough space.
*/

fds->check_for_write_at_end == False;

length = requested_position + len;
(void) fstatvfs(fds->fd, buf);
if (length >= (buf.f_bavail*buf.f_frsize)) {
/* Then the copy will fail eventually, so */
/* report it now. This will avoid NT (Win2K)
*/
/* assuming it can't fail, and then */
/* running out of space. The latter leaves */
/* files full of nulls as the result of a */
/* "successful" copy. */

--dave
--
David Collier-Brown, | Always do right. This will gratify some people
185 Ellerslie Ave., | and astonish the rest. -- Mark Twain
Willowdale, Ontario | //www.oreilly.com/catalog/samba/author.html
Work: (905) 415-2849 Home: (416) 223-8968 Email: ***@canada.sun.com
Nicolas Williams
2003-12-02 09:04:08 UTC
Permalink
Post by Andreas Abach
There is still another bug in Samba. When writing a file to share that
is larger than the free space on the disk the file is allocated with
its full length. During the copy process you'll get an error, but the
file will be left on the samba-share and it will be reported with it's
original size. A full report can be read at
www.lkt.uni-erlangen.de/samba-bug/. The same happens when quotas are
used. I send this problem to this list some weeks ago, but it seams to
unimportant to the mass.
It's not unimportant, it's just unclear how to
fix it ! The client is expliticly setting the file
size, then one of the writes into the file is failing
(probably with ENOSPC). Now what should Samba do here ?
The problem is that on Unix systems seeking a file descriptor position
over file space that's not been allocated and writing any data at that
point causes "holey" files.

Obviously this must not be the case in the Windows world.

Thus, the most complete fix for this problem, IMNSHO, would be to write
the implicit nulls when a client asks Samba to do so. This approach
would result in pre-allocation of all the blocks necessary and would
fail before completing if the file system in question fills up before
the operation can be completed. This will result in just the behaviour
the clients expect.
The client *explicitly* told us what file size to set.
Do we then override that and deliberately truncate the
file on write fail ? Should we do this ? Opinions
please.
See above.
Thanks,
Jeremy Allison,
Samba Team.
Nico
-DISCLAIMER: an automatically appended disclaimer may follow. By posting-
-to a public e-mail mailing list I hereby grant permission to distribute-
-and copy this message.-

This message contains confidential information and is intended only
for the individual named. If you are not the named addressee you
should not disseminate, distribute or copy this e-mail. Please
notify the sender immediately by e-mail if you have received this
e-mail by mistake and delete this e-mail from your system.

E-mail transmission cannot be guaranteed to be secure or error-free
as information could be intercepted, corrupted, lost, destroyed,
arrive late or incomplete, or contain viruses. The sender therefore
does not accept liability for any errors or omissions in the contents
of this message which arise as a result of e-mail transmission. If
verification is required please request a hard-copy version. This
message is provided for informational purposes and should not be
construed as a solicitation or offer to buy or sell any securities or
related financial instruments.
Nicolas Williams
2003-12-02 09:04:08 UTC
Permalink
Post by David Collier-Brown
typedef struct file_fd_struct {
...
BOOL check_for_write_at_end;
} file_fd_struct;
in smbd/open.c open_file()
fds->check_for_write_at_end = True;
...
in <the initial write>
if ( fds->check_for_write_at_end == True
&& requested_position != 0) {
/* Then application was writing the last block */
/* of the file first to see if we have enough space.
*/
fds->check_for_write_at_end == False;
length = requested_position + len;
(void) fstatvfs(fds->fd, buf);
if (length >= (buf.f_bavail*buf.f_frsize)) {
/* Then the copy will fail eventually, so */
/* report it now. This will avoid NT (Win2K)
*/
/* assuming it can't fail, and then */
/* running out of space. The latter leaves */
/* files full of nulls as the result of a */
Well, I think that the 'check_for_write_at_end' flag should be checked
only if the write fails, then do something (such as truncation the file
to 0, to the place where the write failed or unlink the file).

Alternatively see my previous post, where I suggest that Samba prevent
the creation of holey files by SMB clients in the first place.
I may be away, but through the magic of the internet
(and the Computing in High Energy Physics conference :-) I'm still trying
to get work done :-) :-).
We could prbably do a simpler (and nastier :-) fix, by assuming
that if a write fails with ENOSPC, then we should truncate at
the requested seek postion for the file. Hmmm. I can see problems
with this though - and I *hate* the idea of adding the overhead
above to fix what is essentially a client bug...
Even simpler might be the no-holey-files-by-smb-clients fix.
Maybe when a client does a filesize set, we should do the
above space check. Yes - I like that, it's much
cleaner. It also means no special new flag in the fd
struct Dave, just do your space check on the setlength
call and refuse an error if it would be over quota.
Well, the problem with doing the filesystem space check when the client
sets the file size without actually allocating the file blocks is that
there may be enough space at the time of the check, and yet the write
may fail later if there are other clients writing to that partition!!

So, methinks that the filesystem space check at file size set time is
only worthwhile if Samba is also going to preallocate the file blocks as
well. This is probably not a cheap operation though...
What dou you think ?
Jeremy.
Nico
-DISCLAIMER: an automatically appended disclaimer may follow. By posting-
-to a public e-mail mailing list I hereby grant permission to distribute-
-and copy this message.-

This message contains confidential information and is intended only
for the individual named. If you are not the named addressee you
should not disseminate, distribute or copy this e-mail. Please
notify the sender immediately by e-mail if you have received this
e-mail by mistake and delete this e-mail from your system.

E-mail transmission cannot be guaranteed to be secure or error-free
as information could be intercepted, corrupted, lost, destroyed,
arrive late or incomplete, or contain viruses. The sender therefore
does not accept liability for any errors or omissions in the contents
of this message which arise as a result of e-mail transmission. If
verification is required please request a hard-copy version. This
message is provided for informational purposes and should not be
construed as a solicitation or offer to buy or sell any securities or
related financial instruments.
Andreas Abach
2003-12-02 09:04:08 UTC
Permalink
Post by Andreas Abach
There is still another bug in Samba. When writing a file to share that
is larger than the free space on the disk the file is allocated with
its full length. During the copy process you'll get an error, but the
file will be left on the samba-share and it will be reported with it's
original size. A full report can be read at
www.lkt.uni-erlangen.de/samba-bug/. The same happens when quotas are
used. I send this problem to this list some weeks ago, but it seams to
unimportant to the mass.
It's not unimportant, it's just unclear how to
fix it ! The client is expliticly setting the file
size, then one of the writes into the file is failing
(probably with ENOSPC). Now what should Samba do here ?
The client *explicitly* told us what file size to set.
Do we then override that and deliberately truncate the
file on write fail ? Should we do this ? Opinions
please.
If the file is truncated and an error-message on the client will popup,
the user will click on the message and still can see his file. He won't
recognize the problem, even if it would have the correct size (size
after truncating, but smaller than original). So the user will collect
damaged files and we will not understand what is going wrong. So there
should no file be written if it will not fit on the disk or quota.
If this problem is changed, will it work with quota's, too? Or is there
no development in the quota-support at the moment?
I don't understand this quotassituation. Is there nobody who is using
it?

Andreas,
Thanks,
Jeremy Allison,
Samba Team.
--
Dipl.-Ing. Andreas Abach * Lehrstuhl fuer Kunststofftechnik Uni Erlangen
Tel.: +49-9131-85297-06 * Fax.: -09 * email: ***@LKT.Uni-Erlangen.de
Am Weichselgarten 9 * D-91058 Erlangen * www.keramikspritzgiessen.de
PGPID 0x5672B1F8 F-Print FD50 85C0 7580 5467 65E7 3BE8 58BC 65F4 5672 B1F8
David Collier-Brown
2003-12-02 09:04:08 UTC
Permalink
I may be away, but through the magic of the internet
(and the Computing in High Energy Physics conference :-) I'm still trying
to get work done :-) :-).
We could prbably do a simpler (and nastier :-) fix, by assuming
that if a write fails with ENOSPC, then we should truncate at
the requested seek postion for the file. Hmmm. I can see problems
with this though - and I *hate* the idea of adding the overhead
above to fix what is essentially a client bug...
In the Win95 case, the server says ENOSPC and the
client says "delete the file", so we don't need
to worry about 95. Assuming 98 is the same, the
only twitchyness is with NT.
Maybe when a client does a filesize set, we should do the
above space check. Yes - I like that, it's much
cleaner.
What dou you think
Cool: if NT does an explicit set (I haven't seen
logs, so I'd assumed it wrote the last block) then
a check against the quota at that time would do the
job. And it would be portable, in the weak sense that
quotas are portable (;-))

Can someone send me a log of NT being silly?
Preferably with quotas turned on!

--dave
--
David Collier-Brown, | Always do right. This will gratify some people
185 Ellerslie Ave., | and astonish the rest. -- Mark Twain
Willowdale, Ontario | //www.oreilly.com/catalog/samba/author.html
Work: (905) 415-2849 Home: (416) 223-8968 Email: ***@canada.sun.com
Cole, Timothy D.
2003-12-02 09:04:09 UTC
Permalink
-----Original Message-----
Sent: Monday, February 07, 2000 11:30
To: Multiple recipients of list SAMBA-TECHNICAL
Subject: Re: preallocated file size
I may be away, but through the magic of the internet
(and the Computing in High Energy Physics conference :-) I'm still trying
to get work done :-) :-).
We could prbably do a simpler (and nastier :-) fix, by assuming
that if a write fails with ENOSPC, then we should truncate at
the requested seek postion for the file. Hmmm. I can see problems
with this though - and I *hate* the idea of adding the overhead
above to fix what is essentially a client bug...
Maybe when a client does a filesize set, we should do the
above space check. Yes - I like that, it's much
cleaner. It also means no special new flag in the fd
struct Dave, just do your space check on the setlength
call and refuse an error if it would be over quota.
What dou you think ?
Maybe only do this for NT clients.

Under Unix, the actual disk space is not alocated until it is
written to -- consequently, there are a few Unix apps that expect to be able
to get away with seeking and writing way out into a file, even if the
resulting file size would be more than allowed by disk space or quota.

So, this might cause issues for Unix SMB clients.
Cole, Timothy D.
2003-12-02 09:04:09 UTC
Permalink
-----Original Message-----
Sent: Monday, February 07, 2000 12:15
To: Multiple recipients of list SAMBA-TECHNICAL
Subject: Re: preallocated file size
Post by David Collier-Brown
typedef struct file_fd_struct {
...
BOOL check_for_write_at_end;
} file_fd_struct;
in smbd/open.c open_file()
fds->check_for_write_at_end = True;
...
in <the initial write>
if ( fds->check_for_write_at_end == True
&& requested_position != 0) {
/* Then application was writing the last block */
/* of the file first to see if we have enough space.
*/
fds->check_for_write_at_end == False;
length = requested_position + len;
(void) fstatvfs(fds->fd, buf);
if (length >= (buf.f_bavail*buf.f_frsize)) {
/* Then the copy will fail eventually, so */
/* report it now. This will avoid NT (Win2K)
*/
/* assuming it can't fail, and then */
/* running out of space. The latter leaves */
/* files full of nulls as the result of a */
Well, I think that the 'check_for_write_at_end' flag should be checked
only if the write fails, then do something (such as truncation the file
to 0, to the place where the write failed or unlink the file).
Alternatively see my previous post, where I suggest that Samba prevent
the creation of holey files by SMB clients in the first place.
I may be away, but through the magic of the internet
(and the Computing in High Energy Physics conference :-) I'm still
trying
to get work done :-) :-).
We could prbably do a simpler (and nastier :-) fix, by assuming
that if a write fails with ENOSPC, then we should truncate at
the requested seek postion for the file. Hmmm. I can see problems
with this though - and I *hate* the idea of adding the overhead
above to fix what is essentially a client bug...
Even simpler might be the no-holey-files-by-smb-clients fix.
What if the SMB client is a Unix system?
Nicolas Williams
2003-12-02 09:04:09 UTC
Permalink
Post by Cole, Timothy D.
-----Original Message-----
Sent: Monday, February 07, 2000 12:15
To: Multiple recipients of list SAMBA-TECHNICAL
Subject: Re: preallocated file size
Post by David Collier-Brown
typedef struct file_fd_struct {
...
BOOL check_for_write_at_end;
} file_fd_struct;
in smbd/open.c open_file()
fds->check_for_write_at_end = True;
...
in <the initial write>
if ( fds->check_for_write_at_end == True
&& requested_position != 0) {
/* Then application was writing the last block */
/* of the file first to see if we have enough space.
*/
fds->check_for_write_at_end == False;
length = requested_position + len;
(void) fstatvfs(fds->fd, buf);
if (length >= (buf.f_bavail*buf.f_frsize)) {
/* Then the copy will fail eventually, so */
/* report it now. This will avoid NT (Win2K)
*/
/* assuming it can't fail, and then */
/* running out of space. The latter leaves */
/* files full of nulls as the result of a */
Well, I think that the 'check_for_write_at_end' flag should be checked
only if the write fails, then do something (such as truncation the file
to 0, to the place where the write failed or unlink the file).
Alternatively see my previous post, where I suggest that Samba prevent
the creation of holey files by SMB clients in the first place.
I may be away, but through the magic of the internet
(and the Computing in High Energy Physics conference :-) I'm still
trying
to get work done :-) :-).
We could prbably do a simpler (and nastier :-) fix, by assuming
that if a write fails with ENOSPC, then we should truncate at
the requested seek postion for the file. Hmmm. I can see problems
with this though - and I *hate* the idea of adding the overhead
above to fix what is essentially a client bug...
Even simpler might be the no-holey-files-by-smb-clients fix.
What if the SMB client is a Unix system?
What about that? I mean, CIFS doesn't seem to allow for holey-files,
right? Though I imagine the specs say nothing of the subject, so it's
really a matter of practice vs. spec then.

So let the Unix clients use a different protocol. It's not like there
aren't any (Sun is paying for an implementation of NFSv4 for Linux;
NFSv4 appears to be stateful, not just in that it allows for NFS over
TCP, but in that it now has OPEN/CLOSE RPC calls!).

Besides, what's the common case? Windows or Unix SMB clients?

Nico
-DISCLAIMER: an automatically appended disclaimer may follow. By posting-
-to a public e-mail mailing list I hereby grant permission to distribute-
-and copy this message.-

This message contains confidential information and is intended only
for the individual named. If you are not the named addressee you
should not disseminate, distribute or copy this e-mail. Please
notify the sender immediately by e-mail if you have received this
e-mail by mistake and delete this e-mail from your system.

E-mail transmission cannot be guaranteed to be secure or error-free
as information could be intercepted, corrupted, lost, destroyed,
arrive late or incomplete, or contain viruses. The sender therefore
does not accept liability for any errors or omissions in the contents
of this message which arise as a result of e-mail transmission. If
verification is required please request a hard-copy version. This
message is provided for informational purposes and should not be
construed as a solicitation or offer to buy or sell any securities or
related financial instruments.
Nicolas Williams
2003-12-02 09:04:09 UTC
Permalink
Post by Cole, Timothy D.
Post by Nicolas Williams
Even simpler might be the no-holey-files-by-smb-clients fix.
What if the SMB client is a Unix system?
Can Samba detect the type of client? If so then this behaviour might be
made configurable.

Since non-POSIX clients don't expect holey file semantics Samba could
zero-fill what would otherwise be holey files for them. The impact on
performance would not be nice though.

Hmmm, the performance problem might just argue for the truncate-on-no-space
approach instead...

Nico
-DISCLAIMER: an automatically appended disclaimer may follow. By posting-
-to a public e-mail mailing list I hereby grant permission to distribute-
-and copy this message.-

This message contains confidential information and is intended only
for the individual named. If you are not the named addressee you
should not disseminate, distribute or copy this e-mail. Please
notify the sender immediately by e-mail if you have received this
e-mail by mistake and delete this e-mail from your system.

E-mail transmission cannot be guaranteed to be secure or error-free
as information could be intercepted, corrupted, lost, destroyed,
arrive late or incomplete, or contain viruses. The sender therefore
does not accept liability for any errors or omissions in the contents
of this message which arise as a result of e-mail transmission. If
verification is required please request a hard-copy version. This
message is provided for informational purposes and should not be
construed as a solicitation or offer to buy or sell any securities or
related financial instruments.
Cole, Timothy D.
2003-12-02 09:04:09 UTC
Permalink
-----Original Message-----
Sent: Tuesday, February 08, 2000 13:51
To: Multiple recipients of list SAMBA-TECHNICAL
Subject: Re: preallocated file size
Post by Cole, Timothy D.
Post by Nicolas Williams
Even simpler might be the no-holey-files-by-smb-clients fix.
What if the SMB client is a Unix system?
Can Samba detect the type of client? If so then this behaviour might be
made configurable.
From the sound of things, we only need to worry about NT clients,
which (if I understand correctly) are apparently slightly broken, which is
why this makes a difference. If we just default to POSIX semantics
otherwise, it probably wouldn't hurt anything.
jeremy at varesearch.com ()
2003-12-02 09:04:08 UTC
Permalink
A non-text attachment was scrubbed...
Name: not available
Type: text
Size: 928 bytes
Desc: not available
Url : http://lists.samba.org/archive/samba-technical/attachments/20000207/d92d9d07/attachment.bat
jeremy at varesearch.com ()
2003-12-02 09:04:08 UTC
Permalink
A non-text attachment was scrubbed...
Name: not available
Type: text
Size: 2749 bytes
Desc: not available
Url : http://lists.samba.org/archive/samba-technical/attachments/20000207/14f64a50/attachment.bat
Loading...