On loopback device creation

I was having some problems with loop back device creating & almost immediately detaching; it seems sometimes a kernel thread holds on to the device and the detach fails.
I decided to have a look what's going on so I created a script which automates the creation, attaching and detaching of loopback devices:

# ./test.sh

Creating sparse 4TB file: Done
----START----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
----NEXT PASS----
Loopback devices currently in use:
/dev/loop0
Checking for free loopback device: /dev/loop1
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
----NEXT PASS----
Loopback devices currently in use:
/dev/loop0
/dev/loop1
Checking for free loopback device: /dev/loop2
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
----NEXT PASS----
Loopback devices currently in use:
/dev/loop0
/dev/loop1
/dev/loop2
Checking for free loopback device: /dev/loop3
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
----NEXT PASS----
Loopback devices currently in use:
/dev/loop0
/dev/loop1
/dev/loop2
/dev/loop3
Checking for free loopback device: /dev/loop4
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
----NEXT PASS----
Loopback devices currently in use:
/dev/loop0
/dev/loop1
/dev/loop2
/dev/loop3
/dev/loop4
Checking for free loopback device: /dev/loop5
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
----NEXT PASS----
Loopback devices currently in use:
/dev/loop0
/dev/loop1
/dev/loop2
/dev/loop3
/dev/loop4
/dev/loop5
Checking for free loopback device: /dev/loop6
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
----NEXT PASS----
Loopback devices currently in use:
/dev/loop0
/dev/loop1
/dev/loop2
/dev/loop3
/dev/loop4
/dev/loop5
/dev/loop6
Checking for free loopback device: /dev/loop7
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
----NEXT PASS----
Loopback devices currently in use:
/dev/loop0
/dev/loop1
/dev/loop2
/dev/loop3
/dev/loop4
/dev/loop5
/dev/loop6
/dev/loop7
Checking for free loopback device: losetup: could not find any free loop device
Exiting, couldn't find any free device any more.

So if done fast enough the pool quickly gets depleted as giving back (detaching) the device fails. A second or so later it might work, though. But let's test how much time exactly is needed to free the device? I modified the script to keep trying to detach/free the loopback device and see what happens:

Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
Loopback device broken down after 4 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
Loopback device broken down after 2 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
Loopback device broken down after 6 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
loop: can't delete device /dev/loop0: Device or resource busy
Loopback device broken down after 1 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
Loopback device broken down after 3 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
Loopback device broken down after 3 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.

Breaking down loopback device:
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
Loopback device broken down after 2 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
Loopback device broken down after 3 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
Loopback device broken down after 2 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
Loopback device broken down after 4 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
Loopback device broken down after 4 tries.
----NEXT PASS----
Loopback devices currently in use:

Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
loop: can't delete device /dev/loop0: Device or resource busy
Loopback device broken down after 1 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
Loopback device broken down after 5 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
Loopback device broken down after 10 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
Loopback device broken down after 2 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
Loopback device broken down after 4 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
Loopback device broken down after 4 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
Loopback device broken down after 3 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
Loopback device broken down after 0 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
Loopback device broken down after 4 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
Loopback device broken down after 3 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
Loopback device broken down after 2 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
Loopback device broken down after 0 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
loop: can't delete device /dev/loop0: Device or resource busy
Loopback device broken down after 1 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
Loopback device broken down after 3 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
Loopback device broken down after 2 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
loop: can't delete device /dev/loop0: Device or resource busy
Loopback device broken down after 1 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
loop: can't delete device /dev/loop0: Device or resource busy
loop: can't delete device /dev/loop0: Device or resource busy
Loopback device broken down after 2 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
loop: can't delete device /dev/loop0: Device or resource busy
Loopback device broken down after 1 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
loop: can't delete device /dev/loop0: Device or resource busy
Loopback device broken down after 1 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: ^C

So it varies. The device might get released instantaneously, but that doesn't happen too often, or it might take 10 consecutive tries. Let's see if allowing half a second's worth of pause in between creating and detaching fix the problem,

# ./test.sh
Creating sparse 4TB file: Done
----START----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0.5 seconds.
Breaking down loopback device:
Loopback device broken down after 0 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0.5 seconds.
Breaking down loopback device:
Loopback device broken down after 0 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0.5 seconds.
Breaking down loopback device:
Loopback device broken down after 0 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0.5 seconds.
Breaking down loopback device:
Loopback device broken down after 0 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0.5 seconds.

^C

It does, but what's holding on to the device exactly? I modified the script again to look up the did in the process table. This takes time of course, so the detaching phase took less tries to complete:

# ./test.sh
Creating sparse 4TB file: Done
----START----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
loop: can't delete device /dev/loop0: Device or resource busy
root     25417  0.0  0.0      0     0 ?        S<   13:25   0:00 [loop0]
Loopback device broken down after 1 tries.
----NEXT PASS----
Loopback devices currently in use:
Checking for free loopback device: /dev/loop0
Setting up loopback device: Done
Sleeping for 0 seconds.
Breaking down loopback device:
loop: can't delete device /dev/loop0: Device or resource busy
root     25437  0.0  0.0      0     0 ?        S<   13:25   0:00 [loop0]
Loopback device broken down after 1 tries.
----NEXT PASS----
Loopback devices currently in use:
^C

Right, so it's Linux that's holding on to it and I'm not sure if I like this behavior. I realize losetup can only request Linux to free up the device and it does cleanly report a 'Device or resource busy' but I don't understand why it just doesn't wait a bit until the resource is cleared from the kernel, or at least provides the option to the user to do so. I thought about pathing losetup but now I can't be bothered anymore, so I'm just putting it out there :)