I have two files with domain names: input15.txt and input10k.txt.
I want to resolve domain names to ip addresses with scala. The code seems to work for 15 domain names but fails on e.g. 50 domains and it fails for 10000 domains.
ERROR I get:
Resolved(www.ayto-oviedo.es,List(ARecord(www.ayto-oviedo.es,Ttl(30 seconds),www.ayto-oviedo.es/154.58.19.170)),List())
Resolved(www.arianapassion.com,List(ARecord(www.arianapassion.com,Ttl(30 seconds),www.arianapassion.com/107.149.32.132)),List())
Resolved(www.mpa-ppm.org,List(ARecord(www.mpa-ppm.org,Ttl(30 seconds),www.mpa-ppm.org/139.59.238.128)),List())
Resolved(www.regus.it,List(ARecord(www.regus.it,Ttl(30 seconds),www.regus.it/45.60.198.10)),List())
Resolved(www.myboardcast.com,List(ARecord(www.myboardcast.com,Ttl(30 seconds),www.myboardcast.com/217.160.223.88), AAAARecord(www.myboardcast.com,Ttl(30 seconds),www.myboardcast.com/2001:8d8:1000:3005:107b:8f75:a0ce:4008)),List())
Resolved(www.amanto.be,List(ARecord(www.amanto.be,Ttl(30 seconds),www.amanto.be/134.209.201.37)),List())
Resolved(www.srsportal.com,List(ARecord(www.srsportal.com,Ttl(30 seconds),www.srsportal.com/209.99.17.70)),List())
Resolved(www.staffingcalendar.com,List(ARecord(www.staffingcalendar.com,Ttl(30 seconds),www.staffingcalendar.com/69.20.105.131)),List())
Resolved(www.haikoheinz.de,List(),List())
akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://default/system/IO-DNS#-1532696674]] after [1000 ms]. Message of type [akka.io.dns.DnsProtocol$Resolve]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply.
[INFO] [akkaDeadLetter][01/28/2021 13:30:39.724] [default-akka.actor.default-dispatcher-4] [akka://default/deadLetters] Message [akka.io.dns.DnsProtocol$Resolved] from Actor[akka://default/system/IO-DNS/inet-address/$a#1934138291] to Actor[akka://default/deadLetters] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://default/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
[INFO] [akkaDeadLetter][01/28/2021 13:30:39.825] [default-akka.actor.default-dispatcher-5] [akka://default/deadLetters] Message [akka.io.dns.DnsProtocol$Resolved] from Actor[akka://default/system/IO-DNS/inet-address/$b#-1655801503] to Actor[akka://default/deadLetters] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://default/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
[INFO] [akkaDeadLetter][01/28/2021 13:30:40.018] [default-akka.actor.default-dispatcher-5] [akka://default/deadLetters] Message [akka.io.dns.DnsProtocol$Resolved] from Actor[akka://default/system/IO-DNS/inet-address/$c#1227157498] to Actor[akka://default/deadLetters] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://default/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
[INFO] [akkaDeadLetter][01/28/2021 13:30:40.132] [default-akka.actor.default-dispatcher-5] [akka://default/deadLetters] Message [akka.io.dns.DnsProtocol$Resolved] from Actor[akka://default/system/IO-DNS/inet-address/$a#1934138291] to Actor[akka://default/deadLetters] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://default/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
[INFO] [akkaDeadLetter][01/28/2021 13:30:40.142] [default-akka.actor.default-dispatcher-4] [akka://default/deadLetters] Message [akka.io.dns.DnsProtocol$Resolved] from Actor[akka://default/system/IO-DNS/inet-address/$b#-1655801503] to Actor[akka://default/deadLetters] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://default/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
[INFO] [akkaDeadLetter][01/28/2021 13:30:40.248] [default-akka.actor.default-dispatcher-5] [akka://default/deadLetters] Message [akka.io.dns.DnsProtocol$Resolved] from Actor[akka://default/system/IO-DNS/inet-address/$b#-1655801503] to Actor[akka://default/deadLetters] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://default/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
[INFO] [akkaDeadLetter][01/28/2021 13:30:40.261] [default-akka.actor.default-dispatcher-4] [akka://default/deadLetters] Message [akka.io.dns.DnsProtocol$Resolved] from Actor[akka://default/system/IO-DNS/inet-address/$a#1934138291] to Actor[akka://default/deadLetters] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://default/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
[INFO] [akkaDeadLetter][01/28/2021 13:30:40.302] [default-akka.actor.default-dispatcher-4] [akka://default/deadLetters] Message [akka.io.dns.DnsProtocol$Resolved] from Actor[akka://default/system/IO-DNS/inet-address/$c#1227157498] to Actor[akka://default/deadLetters] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://default/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
[INFO] [akkaDeadLetter][01/28/2021 13:30:40.303] [default-akka.actor.default-dispatcher-4] [akka://default/deadLetters] Message [akka.io.dns.DnsProtocol$Resolved] from Actor[akka://default/system/IO-DNS/inet-address/$c#1227157498] to Actor[akka://default/deadLetters] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://default/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
[INFO] [akkaDeadLetter][01/28/2021 13:30:40.424] [default-akka.actor.default-dispatcher-4] [akka://default/deadLetters] Message [akka.io.dns.DnsProtocol$Resolved] from Actor[akka://default/system/IO-DNS/inet-address/$b#-1655801503] to Actor[akka://default/deadLetters] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://default/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
single domain is a string.
I use resources configuration:
resources.conf
akka.io.dns {
resolver = "async-dns"
async-dns {
nameservers = ["8.8.8.8"]
resolve-ipv4 = true
resolve-ipv6 = false
resolve-timeout = 1s
}
}
my code:
import akka.NotUsed
import akka.actor.ActorSystem
import akka.io.{Dns, IO}
import akka.io.dns.DnsProtocol
import akka.pattern.ask
import akka.util.Timeout
import scala.concurrent.duration._
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.Future
import scala.io.{Source => fileSource}
import akka.stream.scaladsl.{Source => strmSource}
object ProxyResolver {
implicit val system = ActorSystem()
implicit val timeout = Timeout(1.second) // Timeout causes Futures to timeout
def main(args: Array[String]): Unit = {
val data = fileSource.fromResource("data/input50.txt").getLines()
val source: strmSource[String, NotUsed] = strmSource.fromIterator(() => data)
getResolved(source)
}
def getResolved(source: strmSource[String, NotUsed]): Unit = {
source
.mapAsync(100) { domain =>
val resolved: Future[DnsProtocol.Resolved] = (IO(Dns) ? DnsProtocol.Resolve(domain)).mapTo[DnsProtocol.Resolved]
resolved
}
.runForeach(println).failed.foreach(println)
}
}
my build.sbt
name := "proxy_resolver"
version := "0.1"
scalaVersion := "2.12.12"
val akkaVersion = "2.6.10"
libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-stream" % akkaVersion,
"com.typesafe.akka" %% "akka-actor" % akkaVersion,
)
EDIT: How to avoid Timeout?
question from:https://stackoverflow.com/questions/65937181/scala-akka-async-dns-with-collection-of-strings
