VB.NET 中使用 WebRequest 和 StreamWriter 的多线程

Question

我需要让它在线程中工作，例如，线程 1 使用 'order_id=1' 调用 url，线程 2 使用 'order_id=2' 调用 url，依此类推, 然后将结果写入文件。

代码如下：

Public Sub download()
    Dim address As String = "http://www.example.com/sales.php&order_id="
    Dim FILE_NAME As String = "D:\Folder\Licenses.txt"
    Dim index As Integer = 0

    Do While index <= 100
            If index > 100 Then
                Exit Do
            End If
        Try
            Dim request As WebRequest = WebRequest.Create(address & index.ToString)
            Dim response As WebResponse = request.GetResponse()                
            If CType(response, HttpWebResponse).StatusDescription = "OK" Then
                Dim dataStream As Stream = response.GetResponseStream()
                Dim reader As New StreamReader(dataStream)
                Dim responseFromServer As String = reader.ReadToEnd()
                If Not File.Exists(FILE_NAME) Then
                    Using sw As StreamWriter = File.CreateText(FILE_NAME)
                        sw.WriteLine(responseFromServer)
                        index += 1
                    End Using
                ElseIf File.Exists(FILE_NAME) Then
                    Using sw As StreamWriter = File.AppendText(FILE_NAME)
                        sw.WriteLine(responseFromServer)
                        index += 1
                    End Using
                End If
            End If
        Catch ex As Exception
        End Try
        Loop
End Sub

Answer 1

一个非常简单的方法是使用 Parallel.For()。您只需确保不要同时向输出文件写入不同的内容。这是通过 Monitor 解决的，它确保临界区中只有一个线程。省略异常处理：

Dim address As String = "http://www.example.com/sales.php&order_id="
Dim FILE_NAME As String = "D:\Folder\Licenses.txt"

Using fstream As New StreamWriter(FILE_NAME)
    Parallel.For(0, 100, Sub(i As Integer)
                           Dim client As New WebClient
                           Dim content = client.DownloadString(address + i.ToString())
                           Monitor.Enter(fstream)
                           fstream.WriteLine(content)
                           Monitor.Exit(fstream)
                       End Sub)
End Using

可以通过将 Web 客户端创建为线程本地对象来改进此示例。这将避免为每次迭代创建新客户端。

Answer 2

我会尝试查看 Microsoft 的 Reactive Framework（NuGet "System.Reactive" 或 "Rx-Main" 以获取位）。

那么你可以这样做：

Public Sub download()

    Dim address As String = "http://www.example.com/sales.php&order_id="
    Dim FILE_NAME As String = "D:\Folder\Licenses.txt"

    Dim download As Func(Of Integer, String) = _
        Function(n)
            Using wc As New WebClient()
                Return wc.DownloadString(New Uri(address & n.ToString()))
            End Using
        End Function

    Dim query = _
        From index In Observable.Range(0, 101) _
        From result In Observable.Start(Function () download(index))
        Select New With {.Index = index, .Result = result}

    Dim results = _
        query _
            .ToArray() _
            .Select(Function(xs) xs _
                .OrderBy(Function(x) x.Index) _
                .Select(Function(x) x.Result) _
                .ToArray())

    results.Do(Sub(lines) File.WriteAllLines(FILE_NAME, lines)).Wait()

End Sub

这将异步查询每个 URL，然后在完成后执行单个文件写入。

VB.NET 中使用 WebRequest 和 StreamWriter 的多线程

Multi-threading with WebRequest and StreamWriter in VB.NET

vb.net

multithreading

streamwriter