开发者

'MaxDegreeOfParallelism' for Array.Parallel?

开发者 https://www.devze.com 2022-12-30 01:36 出处:网络
Is it possible to set \'MaxDegreeOfParallelism\' (that is maximum number of threads to use) for Array.Parallel module since under the hood it u开发者_运维问答ses Parallel.For?According to this post, i

Is it possible to set 'MaxDegreeOfParallelism' (that is maximum number of threads to use) for Array.Parallel module since under the hood it u开发者_运维问答ses Parallel.For?


According to this post, it seems that there is no way to limit the number of threads globally in the final version of Parallel Extensions. An alternative to what brian suggests would be to use PLINQ (which works with parallel sequences) instead of functions that work with arrays.

This can be done using the PSeq module from F# PowerPack. It provides functions such as PSeq.map, PSeq.filter and many other that work with parallel sequences (which can be also nicely composed using pipelining). For parallel sequences, you can use the WithDegreeOfParallelism extension method to specify the behavior.

You could implement a wrapper function for it:
[EDIT: It is already there!]

let withDegreeOfParallelism n (pq:ParallelQuery<_>) = 
  pq.WithDegreeOfParallelsm(n)

And then write:

let res = 
  data |> PSeq.map (fun n -> ...)
       |> PSeq.withDegreeOfParallelism ParallelOptions.MaxDegreeOfParallelism
       |> Array.ofSeq

This may have different perfromance, because it is implemented differently than functions in the Array.Parallel module, but this certainly depends on your scenario.


No, I don't think so.

You can always create your own versions of any of the methods in the Array.Parallel module, using the source code from array.fs (in the CTP release) as a starter.


Assuming I want say at most 10 threads I've been replacing:

myArray 
|> Array.Parallel.iter (fun item -> doWork item)

with

let maxPara = 10
myArray
|> Array.splitInto maxPara
|> Array.Parallel.iter (fun items -> items |> List.iter (fun item -> doWork item))
0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号