Flush 和 Write 调用的特定组合破坏了通过 OpenWrite 使用的 BlockBlob
Specific combinations of Flush and Write calls corrupts BlockBlob used via OpenWrite
我最近遇到了一个错误,或者至少是通过 .NET
客户端使用的 BlockBlob 的一个非常奇怪的行为。
基本上,写入和刷新的特定组合会损坏文件,如果您在写入后读回它,则响应流包含垃圾(更准确地说,它是截断和
最后写入字节的重复)。
重现(100% 机会):
- 程序集Azure.Storage.Blobs,版本=12.8.1.0,文化=中性,PublicKeyToken=92742159e12e44c8
- 使用 BlockBlobClient
- 使用 OpenWrite 获取可写流
- 写入 4 个字节(一次全部写入或一个字节写入)
- 在流上调用 Flush
- 再写 2 个字节(一次全部或一个一个地写入)
- 处理流
- 回读为什么OpenRead
- 看到书面和阅读版本不同
腐败的例子:
var iterations = new List<byte[]?>()
{
new byte[] { 0x00 },
new byte[] { 0x00 },
new byte[] { 0x00 },
new byte[] { 0x00 },
null,
new byte[] { 0xFF },
new byte[] { 0xAA },
};
written: 00000000FFAA
read: FFAAFFAA
成功范例:
var iterations = new List<byte[]?>()
{
new byte[] { 0x00 },
new byte[] { 0x00 },
new byte[] { 0x00 },
null,
new byte[] { 0x00 },
new byte[] { 0xFF },
new byte[] { 0xAA },
};
written: 00000000FFAA
read: 00000000FFAA
null
是我调用 Flush 的点
- 内容可以是任何内容 - 甚至所有迭代都为零
- 0xFF, 0xAA 仅用于突出重复
这里最奇怪的是,如果您在损坏后再写入 1 个字节,那么损坏就会消失。仅当您的 blob 文件以 write4+flush+write2
序列结尾时才会发生。
有人看到这样的东西吗?
完整代码
var containerClient = new BlobContainerClient(connectionSting, "test");
var blockBlobClient = containerClient.GetBlockBlobClient($"test{Guid.NewGuid():N}.data");
var iterations = new List<byte[]?>()
{
new byte[] { 0x00 },
new byte[] { 0x00 },
new byte[] { 0x00 },
new byte[] { 0x00 },
null,
new byte[] { 0xFF },
new byte[] { 0xAA },
};
var writeStream = blockBlobClient.OpenWrite(true, new Azure.Storage.Blobs.Models.BlockBlobOpenWriteOptions()
{
HttpHeaders = new Azure.Storage.Blobs.Models.BlobHttpHeaders()
{
ContentType = "application/octet-stream"
}
});
var writeMemStream = new MemoryStream();
for (var i = 0; i < iterations.Count; i++)
{
var buffer = iterations[i];
if (buffer is null)
{
writeStream.Flush();
writeMemStream.Flush();
}
else
{
writeStream.Write(buffer, 0, buffer.Length);
writeMemStream.Write(buffer, 0, buffer.Length);
}
}
writeStream.Dispose();
var readMemStream = new MemoryStream();
var readStream = blockBlobClient.OpenRead(options: new Azure.Storage.Blobs.Models.BlobOpenReadOptions(false)
{
Position = 0
});
readStream.CopyTo(readMemStream);
readStream.Dispose();
readMemStream.Position = 0;
writeMemStream.Position = 0;
var readArr = readMemStream.ToArray();
var writeArr = writeMemStream.ToArray();
if (!readArr.SequenceEqual(writeArr))
{
System.Console.WriteLine("DIFFERENT");
System.Console.WriteLine(string.Join("", writeArr.Select(q => q.ToString("X2"))));
System.Console.WriteLine(string.Join("", readArr.Select(q => q.ToString("X2"))));
}
else
{
System.Console.WriteLine("SAME");
System.Console.WriteLine(string.Join("", writeArr.Select(q => q.ToString("X2"))));
System.Console.WriteLine(string.Join("", readArr.Select(q => q.ToString("X2"))));
}
我最近遇到了一个错误,或者至少是通过 .NET
客户端使用的 BlockBlob 的一个非常奇怪的行为。
基本上,写入和刷新的特定组合会损坏文件,如果您在写入后读回它,则响应流包含垃圾(更准确地说,它是截断和 最后写入字节的重复)。
重现(100% 机会):
- 程序集Azure.Storage.Blobs,版本=12.8.1.0,文化=中性,PublicKeyToken=92742159e12e44c8
- 使用 BlockBlobClient
- 使用 OpenWrite 获取可写流
- 写入 4 个字节(一次全部写入或一个字节写入)
- 在流上调用 Flush
- 再写 2 个字节(一次全部或一个一个地写入)
- 处理流
- 回读为什么OpenRead
- 看到书面和阅读版本不同
腐败的例子:
var iterations = new List<byte[]?>()
{
new byte[] { 0x00 },
new byte[] { 0x00 },
new byte[] { 0x00 },
new byte[] { 0x00 },
null,
new byte[] { 0xFF },
new byte[] { 0xAA },
};
written: 00000000FFAA
read: FFAAFFAA
成功范例:
var iterations = new List<byte[]?>()
{
new byte[] { 0x00 },
new byte[] { 0x00 },
new byte[] { 0x00 },
null,
new byte[] { 0x00 },
new byte[] { 0xFF },
new byte[] { 0xAA },
};
written: 00000000FFAA
read: 00000000FFAA
null
是我调用 Flush 的点
- 内容可以是任何内容 - 甚至所有迭代都为零
- 0xFF, 0xAA 仅用于突出重复
这里最奇怪的是,如果您在损坏后再写入 1 个字节,那么损坏就会消失。仅当您的 blob 文件以 write4+flush+write2
序列结尾时才会发生。
有人看到这样的东西吗?
完整代码
var containerClient = new BlobContainerClient(connectionSting, "test");
var blockBlobClient = containerClient.GetBlockBlobClient($"test{Guid.NewGuid():N}.data");
var iterations = new List<byte[]?>()
{
new byte[] { 0x00 },
new byte[] { 0x00 },
new byte[] { 0x00 },
new byte[] { 0x00 },
null,
new byte[] { 0xFF },
new byte[] { 0xAA },
};
var writeStream = blockBlobClient.OpenWrite(true, new Azure.Storage.Blobs.Models.BlockBlobOpenWriteOptions()
{
HttpHeaders = new Azure.Storage.Blobs.Models.BlobHttpHeaders()
{
ContentType = "application/octet-stream"
}
});
var writeMemStream = new MemoryStream();
for (var i = 0; i < iterations.Count; i++)
{
var buffer = iterations[i];
if (buffer is null)
{
writeStream.Flush();
writeMemStream.Flush();
}
else
{
writeStream.Write(buffer, 0, buffer.Length);
writeMemStream.Write(buffer, 0, buffer.Length);
}
}
writeStream.Dispose();
var readMemStream = new MemoryStream();
var readStream = blockBlobClient.OpenRead(options: new Azure.Storage.Blobs.Models.BlobOpenReadOptions(false)
{
Position = 0
});
readStream.CopyTo(readMemStream);
readStream.Dispose();
readMemStream.Position = 0;
writeMemStream.Position = 0;
var readArr = readMemStream.ToArray();
var writeArr = writeMemStream.ToArray();
if (!readArr.SequenceEqual(writeArr))
{
System.Console.WriteLine("DIFFERENT");
System.Console.WriteLine(string.Join("", writeArr.Select(q => q.ToString("X2"))));
System.Console.WriteLine(string.Join("", readArr.Select(q => q.ToString("X2"))));
}
else
{
System.Console.WriteLine("SAME");
System.Console.WriteLine(string.Join("", writeArr.Select(q => q.ToString("X2"))));
System.Console.WriteLine(string.Join("", readArr.Select(q => q.ToString("X2"))));
}