开发者

Why does my fork/join deadlock?

开发者 https://www.devze.com 2023-04-08 09:59 出处:网络
Consider the following snipped of code, which calculates the size of all paths given. def pathSizes = []

Consider the following snipped of code, which calculates the size of all paths given.

def pathSizes = []
paths.each { rootPath ->
  pathSizes.addAll(
    withPool { pool ->
      runForkJoin(rootPath) { path ->
        def headSizes = [:]
        println path

        def lines = ["ls", "-al", path].execute().text.readLines()
        (0..<3).each { lines.remove(0) }
        lines.each { line ->
          def fields = line.split(/\s+/)
          if (fields[0] =~ /^d/)
            forkOffChild("$path/${fields.last()}")
          else {
            def userName = fields[2]
            def fileSize = fields[4] as long
            if (headSizes[userName] == null)
              headSizes[userName] = fileSize
            else
              headSizes[userName] += fileSize
          }
        }

        quietlyJoin()
        System.gc()

        def shallowSizes =
          headSizes.collectEntries { userName, fileSize ->
            def childResult =
              childrenResults.sum {
                 it.shallowSizes[userName] ? it.shall开发者_JS百科owSizes[userName] : 0
              } ?: 0
            return [userName, fileSize + childResult]
          }

        def deepSizes =
          childrenResults.sum { it.deepSizes ?: [] } +
          shallowSizes.collect { userName, fileSize ->
            [userName: userName, path: path, fileSize: fileSize]
          }

        return [shallowSizes: shallowSizes, deepSizes: deepSizes]
      }.deepSizes
    })
}

Why does this snippet of code deadlock? There are no interactions between threads except possibly with the system call and other parts of the Java framework. If the system calls are the problem, then how can I fix it, without removing the system calls (they are slow, hence the need to parallelize)?

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号