在 close(2) 之后调用 fsync(2)
Calling fsync(2) after close(2)
场景：
任务代码（省略错误检查）：
// open, write and close
fd = open(name);
write(fd, buf, len);
close(fd);
< more code here **not** issuing read/writes to name but maybe open()ing it >
// open again and fsync
fd = open(name);
fsync(fd);
系统中没有更多任务同时访问 name。
它是否已定义，更重要的是，它会同步 name 引用的 inode 上可能未完成的写入吗？即，我会在 fsync 之后从文件中读回 buf 吗？
来自 POSIX http://pubs.opengroup.org/onlinepubs/009695399/functions/fsync.html 我会说这似乎是合法的...
谢谢。
5 月 18 日编辑：
感谢您的回答和研究。我（2016 年）向一位 extfs 首席开发人员 (Ted) 提出了这个问题，得到了这个答案：“Posix 不能保证，但实际上它应该适用于大多数
文件系统，包括 ext4。 Posix规范中的关键词是：
fsync() 函数应请求打开文件的所有数据
^^^^^^^^^^^^^^^^^
fildes 命名的描述符将被传输到存储设备
^^^^^^^^^^^^^^^^^^^^^^^^^^^
与 fildes 描述的文件关联。
它不说"all data for the file described by fildes...."它
说 "all data for the open file descriptor"。所以技术上的数据
不保证由另一个文件描述符写入的同步到
磁盘。
实际上，文件系统不会尝试通过 fd 传入的脏数据
上，所以你不必担心。还有一个 OS 写的比什么还多
是严格要求的是符合标准的，所以这就是你
一般情况下会找到，即使不能保证。”这不如 "exact same durabily guarrantees" 具体，但相当权威，即使可能已过时。
我试图做的是对单个文件起作用的 'sync' 命令。
就像 fsync /some/file 一样，无需同步整个文件系统，例如在 shell 脚本中使用它。
现在（从几年前开始）gnu coreutils 'sync' 在单个文件上工作并且正是这样做的 (open/fsync)。提交：https://github.com/coreutils/coreutils/commit/8b2bf5295f353016d4f5e6a2317d55b6a8e7fd00
No, close()+re-open()+fsync() 不提供与 fsync()+[ 相同的保证=10=].
来源：我took this question to the linux-fsdevel mailing list and got the answer:
Does a sequence of close()/re-open()/fsync() provide the same durability
guarantees as fsync()/close()?
简短的回答是否定的，后者提供了更好的保证。
更长的答案是持久性保证取决于内核版本，
因为情况在 v4.13、v4.14 和现在再次发生变化
v4.17-rc 和稳定内核。
更多相关链接是：
- https://wiki.postgresql.org/wiki/Fsync_Errors ("fsyncgate")
- 邮件列表条目 PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS
- Writing programs to cope with I/O errors causing lost writes on Linux 来自同一作者
特别是后面的链接描述了如何
- 关闭 FD 后，您将失去所有增强耐久性的方法
- 在
fsync() 失败后，您不能再次调用 fsync() 以希望现在您的数据将被写入
- 你 must re-do/confirm all writing work 如果发生这种情况
POSIX fsync() 当前 (2017) 规范
识别基本功能和可选功能：
The fsync() function shall request that all data for the open file descriptor named by fildes is to be transferred to the storage device associated with the file described by fildes. The nature of the transfer is implementation-defined. The fsync() function shall not return until the system has completed that action or until an error is detected.
[SIO] ⌦ If _POSIX_SYNCHRONIZED_IO is defined, the fsync() function shall force all currently queued I/O operations associated with the file indicated by file descriptor fildes to the synchronized I/O completion state. All I/O operations shall be completed as defined for synchronized I/O file integrity completion. ⌫
如果实现未定义 _POSIX_SYNCHRONIZED_IO，则您重新打开的文件描述符没有未写入的数据要传输到存储设备，因此 fsync() 调用实际上是空操作。
如果实现定义了 _POSIX_SYNCHRONIZED_IO，那么您重新打开的文件描述符将确保写入与要传输到存储设备的文件关联的任何文件描述符上的所有数据。
Conformance 上的标准部分包含有关选项和选项组的信息。
Definitions section has definitions 382..387 定义了 Synchronized I/O 和 Synchronous I/O 的各个方面（是的，它们是不同的——还要注意打开文件描述符和打开文件描述）。
Realtime 部分遵循同步 I/O 含义的定义部分。
它定义了：
3.382 Synchronized Input and Output
A determinism and robustness improvement mechanism to enhance the data input and output mechanisms, so that an application can ensure that the data being manipulated is physically present on secondary mass storage devices.
3.383 Synchronized I/O Completion
The state of an I/O operation that has either been successfully transferred or diagnosed as unsuccessful.
3.384 Synchronized I/O Data Integrity Completion
For read, when the operation has been completed or diagnosed if unsuccessful. The read is complete only when an image of the data has been successfully transferred to the requesting process. If there were any pending write requests affecting the data to be read at the time that the synchronized read operation was requested, these write requests are successfully transferred prior to reading the data.
For write, when the operation has been completed or diagnosed if unsuccessful. The write is complete only when the data specified in the write request is successfully transferred and all file system information required to retrieve the data is successfully transferred.
File attributes that are not necessary for data retrieval (access time, modification time, status change time) need not be successfully transferred prior to returning to the calling process.
3.385 Synchronized I/O File Integrity Completion
Identical to a synchronized I/O data integrity completion with the addition that all file attributes relative to the I/O operation (including access time, modification time, status change time) are successfully transferred prior to returning to the calling process.
3.386 Synchronized I/O Operation
An I/O operation performed on a file that provides the application assurance of the integrity of its data and files.
3.387 Synchronous I/O Operation
An I/O operation that causes the thread requesting the I/O to be blocked from further use of the processor until that I/O operation completes.
Note:
A synchronous I/O operation does not imply synchronized I/O data integrity completion or synchronized I/O file integrity completion.
'all currently queued I/O operations associated with the file indicated by [the] file descriptor' 是否跨进程应用还不是 100% 清楚。
从概念上讲，我认为它应该，但措辞不是黑白的（或淡黄色上的黑色）。它当然应该适用于当前进程中引用同一文件的任何打开的文件描述符。目前尚不清楚它是否适用于当前进程中先前打开（和关闭）的文件描述符。如果它适用于所有进程，那么它应该包括来自当前进程的排队 I/O。如果它不适用于所有进程，则可能不适用。
鉴于此和 fsync() 的基本原理说明，目前最安全的假设是 fsync() 操作对与已关闭文件描述符关联的排队操作没有影响。如果你想让fsync()生效，在关闭文件描述符之前调用它。
场景：
任务代码（省略错误检查）：
// open, write and close
fd = open(name);
write(fd, buf, len);
close(fd);
< more code here **not** issuing read/writes to name but maybe open()ing it >
// open again and fsync
fd = open(name);
fsync(fd);
系统中没有更多任务同时访问 name。
它是否已定义，更重要的是，它会同步 name 引用的 inode 上可能未完成的写入吗？即，我会在 fsync 之后从文件中读回 buf 吗？
来自 POSIX http://pubs.opengroup.org/onlinepubs/009695399/functions/fsync.html 我会说这似乎是合法的...
谢谢。
5 月 18 日编辑： 感谢您的回答和研究。我（2016 年）向一位 extfs 首席开发人员 (Ted) 提出了这个问题，得到了这个答案：“Posix 不能保证，但实际上它应该适用于大多数 文件系统，包括 ext4。 Posix规范中的关键词是：
fsync() 函数应请求打开文件的所有数据 ^^^^^^^^^^^^^^^^^ fildes 命名的描述符将被传输到存储设备 ^^^^^^^^^^^^^^^^^^^^^^^^^^^ 与 fildes 描述的文件关联。
它不说"all data for the file described by fildes...."它 说 "all data for the open file descriptor"。所以技术上的数据 不保证由另一个文件描述符写入的同步到 磁盘。
实际上，文件系统不会尝试通过 fd 传入的脏数据 上，所以你不必担心。还有一个 OS 写的比什么还多 是严格要求的是符合标准的，所以这就是你 一般情况下会找到，即使不能保证。”这不如 "exact same durabily guarrantees" 具体，但相当权威，即使可能已过时。
我试图做的是对单个文件起作用的 'sync' 命令。 就像 fsync /some/file 一样，无需同步整个文件系统，例如在 shell 脚本中使用它。 现在（从几年前开始）gnu coreutils 'sync' 在单个文件上工作并且正是这样做的 (open/fsync)。提交：https://github.com/coreutils/coreutils/commit/8b2bf5295f353016d4f5e6a2317d55b6a8e7fd00
No, close()+re-open()+fsync() 不提供与 fsync()+[ 相同的保证=10=].
来源：我took this question to the linux-fsdevel mailing list and got the answer:
Does a sequence of close()/re-open()/fsync() provide the same durability guarantees as fsync()/close()?
简短的回答是否定的，后者提供了更好的保证。 更长的答案是持久性保证取决于内核版本， 因为情况在 v4.13、v4.14 和现在再次发生变化 v4.17-rc 和稳定内核。
更多相关链接是：
- https://wiki.postgresql.org/wiki/Fsync_Errors ("fsyncgate")
- 邮件列表条目 PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS
- Writing programs to cope with I/O errors causing lost writes on Linux 来自同一作者
特别是后面的链接描述了如何
- 关闭 FD 后，您将失去所有增强耐久性的方法
- 在
fsync()失败后，您不能再次调用fsync()以希望现在您的数据将被写入 - 你 must re-do/confirm all writing work 如果发生这种情况
POSIX fsync() 当前 (2017) 规范
识别基本功能和可选功能：
The
fsync()function shall request that all data for the open file descriptor named byfildesis to be transferred to the storage device associated with the file described byfildes. The nature of the transfer is implementation-defined. Thefsync()function shall not return until the system has completed that action or until an error is detected.[SIO] ⌦ If
_POSIX_SYNCHRONIZED_IOis defined, thefsync()function shall force all currently queued I/O operations associated with the file indicated by file descriptorfildesto the synchronized I/O completion state. All I/O operations shall be completed as defined for synchronized I/O file integrity completion. ⌫
如果实现未定义 _POSIX_SYNCHRONIZED_IO，则您重新打开的文件描述符没有未写入的数据要传输到存储设备，因此 fsync() 调用实际上是空操作。
如果实现定义了 _POSIX_SYNCHRONIZED_IO，那么您重新打开的文件描述符将确保写入与要传输到存储设备的文件关联的任何文件描述符上的所有数据。
Conformance 上的标准部分包含有关选项和选项组的信息。 Definitions section has definitions 382..387 定义了 Synchronized I/O 和 Synchronous I/O 的各个方面（是的，它们是不同的——还要注意打开文件描述符和打开文件描述）。 Realtime 部分遵循同步 I/O 含义的定义部分。
它定义了：
3.382 Synchronized Input and Output
A determinism and robustness improvement mechanism to enhance the data input and output mechanisms, so that an application can ensure that the data being manipulated is physically present on secondary mass storage devices.
3.383 Synchronized I/O Completion
The state of an I/O operation that has either been successfully transferred or diagnosed as unsuccessful.
3.384 Synchronized I/O Data Integrity Completion
For read, when the operation has been completed or diagnosed if unsuccessful. The read is complete only when an image of the data has been successfully transferred to the requesting process. If there were any pending write requests affecting the data to be read at the time that the synchronized read operation was requested, these write requests are successfully transferred prior to reading the data.
For write, when the operation has been completed or diagnosed if unsuccessful. The write is complete only when the data specified in the write request is successfully transferred and all file system information required to retrieve the data is successfully transferred.
File attributes that are not necessary for data retrieval (access time, modification time, status change time) need not be successfully transferred prior to returning to the calling process.
3.385 Synchronized I/O File Integrity Completion
Identical to a synchronized I/O data integrity completion with the addition that all file attributes relative to the I/O operation (including access time, modification time, status change time) are successfully transferred prior to returning to the calling process.
3.386 Synchronized I/O Operation
An I/O operation performed on a file that provides the application assurance of the integrity of its data and files.
3.387 Synchronous I/O Operation
An I/O operation that causes the thread requesting the I/O to be blocked from further use of the processor until that I/O operation completes.
Note: A synchronous I/O operation does not imply synchronized I/O data integrity completion or synchronized I/O file integrity completion.
'all currently queued I/O operations associated with the file indicated by [the] file descriptor' 是否跨进程应用还不是 100% 清楚。 从概念上讲，我认为它应该，但措辞不是黑白的（或淡黄色上的黑色）。它当然应该适用于当前进程中引用同一文件的任何打开的文件描述符。目前尚不清楚它是否适用于当前进程中先前打开（和关闭）的文件描述符。如果它适用于所有进程，那么它应该包括来自当前进程的排队 I/O。如果它不适用于所有进程，则可能不适用。
鉴于此和 fsync() 的基本原理说明，目前最安全的假设是 fsync() 操作对与已关闭文件描述符关联的排队操作没有影响。如果你想让fsync()生效，在关闭文件描述符之前调用它。