WSL1 lxfs file system does not handle file locking properly

[gitcnd]

Windows Version

Microsoft Windows [Version 10.0.17763.6293]

WSL Version

0.0.0.0

Are you using WSL 1 or WSL 2?

• WSL 2
• WSL 1

Kernel Version

Linux version 4.4.0-17763-Microsoft ([email protected]) (gcc version 5.4.0 (GCC) ) #2268-Microsoft Thu Oct 07 16:36:00 PST 2021

Distro Version

Ubuntu 20.04

Other Software

n/a

Repro Steps

Run this in Ubuntu:

for FN in {1..40};do perl perlshared.pl & done

If the lock file is in the lxfs file system, the answer in green is wrong (not 0) and multiple "Cannot truncate file: Invalid argument" errors appear

If the lock file is elsewhere (e.g. the window C: drive) it works (and is also 10 or more times faster than above)

use strict;
use warnings;
#use Fcntl ':flock';  # Import LOCK_* constants
#no strict 'subs';
my $LOCK_EX=2; # These are required, because the LOCK_* constants are sometimes not numbers, and inconveniently require "no strict 'subs';"
my $LOCK_UN=8;
use JSON::XS;

my $json = JSON::XS->new->canonical;

my $lock_share_file = '/tmp/tiehash.dat'; # linux FS
#my $lock_share_file = '/mnt/c/Users/cnd/Downloads/tiehash.dat'; # windows FS

`touch $lock_share_file` unless(-e $lock_share_file);

sub shread {
    my ($key) = @_;
    my($data,$fh)= _load_from_file();
    return $data->{$key};
}

sub shupdate {
    my ($key, $val, $inc) = @_;
    my($data,$fh)= _load_from_file(1);

    # Update the value for the key
    if($inc) {
      $data->{$key} = ($data->{$key} // 0) + $val;
    } else {
      $data->{$key} = $val;
    }
    my $ret=$data->{$key};
    _save_to_file($data,$fh);
    return $ret;
}

sub shreset {
    open my $fh, '>', $lock_share_file or die "$$ Cannot open $lock_share_file: $!";
    print $fh '{}';
    $fh->close;
}

sub _load_from_file {
    my($staylocked)=@_;
    open my $fh, '+<', $lock_share_file or die "$$ Cannot open $lock_share_file: $!";
    my $data = {};

    flock($fh, 2) or die "$$ Cannot lock: $!";
    my $json_text = do { local $/; <$fh> };
    $data = $json->decode($json_text) if $json_text;
    unless($staylocked){  # LOCK_UN (unlock)
      flock($fh, 8) or die "$$ Cannot unlock: $!";
      $fh->close;
    }
    return($data,$fh);
}

sub _save_to_file {
    my ($data,$fh) = @_;
    unless($fh) {
      open $fh, '+<', $lock_share_file or die "$$ Cannot open $lock_share_file: $!";
    }

    flock($fh, 2) or die "$$ Cannot lock: $!";  # LOCK_EX (exclusive lock for writing)
    truncate($fh, 0) or die "$$ Cannot truncate file: $!";
    seek($fh, 0, 0) or die "$$ Cannot seek: $!";
    print $fh $json->encode($data);
    flock($fh, 8) or die "$$ Cannot unlock: $!";  # LOCK_UN (unlock)
    $fh->close;
}


my $first=0;
my $ntest=4;

# Perform 100 read/write operations
&shupdate('last',"I am $$");
&shupdate("me $$",&shread('foo'));
for my $i (1..$ntest) {
    my $value=&shread('foo');
    if ($value == 0) {
        $first=1;
    }
    &shupdate('foo',1,1) # $hash{foo} = $value + 1;  # Increment the value and store it
}
for my $i (1..$ntest) {
    my $value=&shread('foo');
    &shupdate('foo',-1,1) # $hash{foo} = $value + 1;  # Increment the value and store it
}

sleep(2); # so we can spawn 400 other processes at once, and have them all running together

# Print final value of foo
print "$$ Final value of foo: " . &shread('foo') . "\n";

if($first) {
  sleep(10);
  print "\x1b[32;1m$$ Last before ending: foo=" . &shread('foo') . "\x1b[0m\n"; # this will be wrong if many threads meant that they didn't all end in 10s
}

`

Expected Behavior

Should work the same regardless of the file system the lockfile is placed onto.

Actual Behavior

Fails when the lxfs is used.

Diagnostic Logs

No response

[github-actions]

Logs are required for review from WSL team

If this a feature request, please reply with '/feature'. If this is a question, reply with '/question'.
Otherwise please attach logs by following the instructions below, your issue will not be reviewed unless they are added. These logs will help us understand what is going on in your machine.

How to collect WSL logs

Download and execute collect-wsl-logs.ps1 in an administrative powershell prompt:

Invoke-WebRequest -UseBasicParsing "https://raw.githubusercontent.com/microsoft/WSL/master/diagnostics/collect-wsl-logs.ps1" -OutFile collect-wsl-logs.ps1
Set-ExecutionPolicy Bypass -Scope Process -Force
.\collect-wsl-logs.ps1

The script will output the path of the log file once done.

If this is a networking issue, please use collect-networking-logs.ps1, following the instructions here

Once completed please upload the output files to this Github issue.

Click here for more info on logging
If you choose to email these logs instead of attaching to the bug, please send them to [email protected] with the number of the github issue in the subject, and in the message a link to your comment in the github issue and reply with '/emailed-logs'.

View similar issues

Please view the issues below to see if they solve your problem, and if the issue describes your problem please consider closing this one and thumbs upping the other issue to help us prioritize it!

Closed similar issues:

• fcntl(F_SETLK) for setting byte range locks is broken leading to data corruption (#1927), similarity score: 0.72

Note: You can give me feedback by thumbs upping or thumbs downing this comment.

[gitcnd]

p.s. works fine in native linux using zfs, and also using xfs, and also using ext4.