Very quick note for future reference. Please ignore. Change loglevel in spark Easy peasy, you can do it programatically in the application like: spark.sparkContext.setLogLevel("WARN") Change loglevel in yarn This one took a while to find, you can just run spark-submit while previously exporting this envvar: export YARN_ROOT_LOGGER=${YARN_ROOT_LOGGER:-WARN,RFA} That´s all.

Took me a while to figure it out, but it seems that in golang you cant re-read from an http response. I found here a way to solve it. For debugging purposes, I had to be able to print the raw response as well as decoding it to json, to make sure that the json decoder was decoding properly Basically you can reset the http response body “read state” by doing this: Read more

A few weeks ago I read an article in which Timo Grossenbacher showed how he managed to plot, in my opinion, one of the most beautiful maps I have ever seen. So I went and tried to replicate it. First of all, here is the map. As usual, you can download the code & data included in this post in github. I pushed some earlier versions of the map as well, with different color breaks so you can see the impact that binning has in data visualization. Read more

This post serves two purposes; First, it will hopefully get indexed by google and will help future US citizens living in Spain. Second, it will be yet another example of how bad things are in Spain regarding Open Data, Open Access and Transparency. For those that come looking for that info here it is: 1.TL:DR. How to get your Certificate of Coverage for self employed persons in Spain. This is what you do. Read more

So I was updating the testing suite in one of Tribe’s microservices. This one in particular deals with sending the postbacks to the right endpoints. If you search for testing http requests, chances are you will almost always stumble upon an example like: import ( "log" "net/http" "net/http/httptest" ) ts := httptest.NewServer(http.HandlerFunc(func(w ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { fmt.Fprintln(w, "Hello, client") })) defer ts.Close() res, err := http.Get(ts.URL) if err ! Read more