Technipelago AB

Technipelago Blog Stuff that we learned...


Logging HTTP traffic in Micronaut

Publicerad den 16 Nov 2018 av Göran Ehrsson

If you want to see HTTP traffic between your Micronaut HTTP client and servers you can easily enable logging in logback.xml

 

src/main/resources/logback.xml


    
        
            %d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n
        
    

    
        
    

    


 With this log level you will see both headers and body for HTTP requests and responses. See example below.

14:32:18.251 [nioEventLoopGroup-1-3] DEBUG i.m.http.client.DefaultHttpClient - Sending HTTP Request: POST /login
14:32:18.251 [nioEventLoopGroup-1-3] DEBUG i.m.http.client.DefaultHttpClient - Chosen Server: localhost(9005)
14:32:18.253 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - host: localhost:9005
14:32:18.253 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - connection: close
14:32:18.253 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - content-type: application/json
14:32:18.253 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - content-length: 79
14:32:18.253 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - Request Body
14:32:18.253 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - ----
14:32:18.254 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - {"username":"user","password":"password","identity":"user","secret":"password"}
14:32:18.254 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - ----
14:32:18.676 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - HTTP Client Response Received for Request: POST /login
14:32:18.677 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - Status Code: 200 OK
14:32:18.677 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - Date: Fri, 16 Nov 2018 14:32:18 GMT
14:32:18.677 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - content-type: application/json
14:32:18.677 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - content-length: 567
14:32:18.677 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - connection: close
14:32:18.677 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - Response Body
14:32:18.677 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - ----
14:32:18.678 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - {"username":"user","roles":["ROLE_USER"],"access_token":"eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ1c2Vy...","expires_in":3600,"token_type":"Bearer"}
14:32:18.678 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - ----

Tags: micronaut rest


Micronaut module name changes between RC2 and RC3

Publicerad den 15 Oct 2018 av Göran Ehrsson

Once again Perl came to my rescue!

Many years ago I used Perl to batch update Grails source code.

When Micronaut changed module names between RC2 and RC3 I took the old Perl script from my blog and changed it to add micronaut- in front of all dependencies in build.gradle. I only had 15 build.gradle files to change so I could probably have done it manually, but there's many io.micronaut dependencies in each build file so a script is more convenient and eliminate the risk of typos.

In Micronaut RC2 and earlier a io.micronaut dependency could look like this:

compile "io.micronaut:inject"

In RC3 all module names was prefixed with micronaut- 

compile "io.micronaut:micronaut-inject"

Here is the Perl script that change all io.micronaut:..... lines for you:

update-micronaut-module.pl 

while(<>) {
  if(/(io\.micronaut[^:]*):(.+)/) {
    print "$`$1:micronaut-$2$'";
  } else {
    print $_;
  }
}

Make sure you checked in all build.gradle files to your VCS before continuing.

perl -i update-micronaut-module.pl */build.gradle

-i performs inplace editing, i.e overwrites the file after completion

 

WARNING: Don't runt the script twice, as it will add the micronaut- prefix each time you run it.

 

Tags: micronaut perl


The road to Micronaut

Publicerad den 12 Oct 2018 av Göran Ehrsson

I wanted to evaluate the new Micronaut framework and in the middle of September 2018 I made a bold decision...

During 3 months this summer I developed a proof-of-concept backoffice system with 10 microservices in Spring Cloud and a Vue.js front-end. It was a struggle to get all parts working. I used latest milestone relases of Spring components and that did impose some limitations and pain. I have 3-4 years experience with Spring Cloud but I had to cheat and create hacks and workarounds for things that didn't work so well. I'm sure all these issues can be fixed given enough time, education and when using release builds, but I had a demo deadline to meet.

I finished on time and the demo went great, but if you looked behind the scenes it was not pretty. :)


After the summer I wanted to evaluate the new Micronaut framework and in the middle of September I made a bold decision. Migrate all 10 services to Micronaut to get a feeling for the framework!


Micronaut 1.0.0 was not yet released when I started the migration project. I think I started on 1.0.0.M4, upgraded to RC1, RC2 and sometimes I was running on latest commits on the master branch. It was a bumpy road but I managed to solve all issues (the right way, not workarounds) and move on, much thanks to the amazing Micronaut team at OCI. They answered my questions on Gitter, I created issues on Github and they fixed them in no time. After 3 weeks all 10 services were up and running.

Things that I was struggling with in Spring Cloud, like Security, WebSockets, Apache Kafka, etc. was easy to get going in Micronaut. The most problematic part was service discovery with Eureka. But after Graeme Rocher fixed the last bug it all started to work like a dream.

Service Discovery, Security, WebSockets and Kafka are easy to configure (very little code) and testing HTTP endpoints with security enabled are fun again with Micronaut's declarative HTTP clients.

I'm absolutely sure that the Micronaut framework will have a GR8 future!

 

(a more detailed blog post will follow where I share the experiences gathered during this migration project)

Tags: micronaut spring


Micronaut in a Spring Kafka environment

Publicerad den 25 Sep 2018

Spring Kafka encode message headers in a way that makes it incompatible with Micronaut (1.0.0.M4). This article provide one solution to the problem.

When Spring's Kafka implementation sends a message to Kafka it encode header values as JSON. So a simple string hello is written to the header as "hello". When Micronaut Kafka implementation recieves the message it will read the header value including the double quotes. So you header value is now ""hello"". This is probably not what you expected.

The other way around, when Micronaut sends a message to Kafka it just writes string headers as a sequence of bytes.When Spring Kafka decodes the header it will get a byte array instead of a String. And if the implementation expects a String and do messageHeaders.get("headername", String.class) an exception will be thrown.

If you only allow string values in your Kafka headers there is an easy solution on the Spring side to make it compatible with Micronaut (and other non-spring services). But this solution alters the message format for all messages in you cluster so it may not be an option in your environment.

The proposed solution is to add a custom KafkaHeaderMapper in each Spring service that read/wrie messages to Kafka.

 

    @Bean
    public KafkaHeaderMapper myKafkaHeaderMapper() {
        return new KafkaHeaderMapper() {
            @Override
            public void fromHeaders(MessageHeaders headers, Headers target) {
                for (Map.Entry<String, Object> entry : headers.entrySet()) {
                    target.add(entry.getKey(), entry.getValue().toString().getBytes());
                }
            }

            @Override
            public void toHeaders(Headers source, Map<String, Object> target) {
                for (org.apache.kafka.common.header.Header header : source) {
                    target.put(header.key(), new String(header.value()));
                }
            }
        };
    }

 

Creating the bean is not enough, you must also add a reference to the bean name in application.yml.

spring:
  cloud:
    stream:
      kafka:
        binder:
          headerMapperBeanName: myKafkaHeaderMapper

 

Now the Spring service will expect header values to be just plain byte arrays and it will map to/from String accordingly.

 

Original issue https://github.com/micronaut-projects/micronaut-core/issues/605

Tags: spring micronaut kafka


Custom error response for Spring REST endpoints

Publicerad den 1 Oct 2017

It's easy to customize the error response from REST endpoints in Spring Boot. This blog post explains how to add a custom error code to the response.

The default error response from a Spring Boot REST endpoint looks something like this:

{
    "error": "Not Found", 
    "exception": "com.mycompany.rest.MyException", 
    "message": "A record with value [foo] was not found", 
    "path": "/foo", 
    "status": 404, 
    "timestamp": 1507117452642
}

If we want to include custom information into the error response it's easy. Just add an ErrorAttributes bean to the spring context.

The following code snippet checks if the thrown exception is a custom exception MyException. This exception type contains a code attribute that holds a unique error code. We want to include that code as a specific key in the error response.

@SpringBootApplication
@EnableResourceServer
public class MyApplication {

    public static void main(String[] args) {
        SpringApplication.run(MyApplication.class, args);
    }

    @Bean
    public ErrorAttributes errorAttributes() {
        return new DefaultErrorAttributes() {
            @Override
            public Map getErrorAttributes(RequestAttributes requestAttributes, boolean includeStackTrace) {
                final Map<String, Object> errorAttributes = super.getErrorAttributes(requestAttributes, includeStackTrace);
                // Customize the default entries in errorAttributes by adding the error code.
                Throwable exception = this.getError(requestAttributes);
                if (exception instanceof MyException) {
                    errorAttributes.put("code", ((MyException) exception).getCode());
                }
                return errorAttributes;
            }
        };
    }
}

If a controller method throws MyException the response now looks like this:

{
    "code": "12345",
    "error": "Not Found", 
    "exception": "com.mycompany.rest.MyException", 
    "message": "A record with value [foo] was not found", 
    "path": "/foo", 
    "status": 404, 
    "timestamp": 1507117452642
}

Tags: spring boot rest


Grails 3.2.0 - java.lang.IllegalArgumentException: Incorrect type for parameter [tenantId]

Publicerad den 11 Sep 2016

With Grails 3.2.0 native multi-tenancy has finally arrived in Grails core. This makes Grails a powerful framework for multi-tenant web applications. But these advanced features comes with a price. New kind of problems surface and you most know what you're doing more than ever.

Tested with Grails 3.2.0.RC1

Consider the following code:

Person.groovy:

import grails.gorm.MultiTenant

class Person implements MultiTenant {
    Long tenantId
    String firstName
    String lastName
    
    def beforeValidate() {
        if(tenantId == null) {
            tenantId = Tenants.currentId()
        }
    }
}

 

FailTestSpec.groovy:

Tenants.withId(1) {
    new Person(firstName: "Groovy", lastName: "Grails").save(failOnError: true)
}

This looks ok, doesn't it?

Well, this is what you get:

java.lang.IllegalArgumentException: Incorrect type for parameter [tenantId]

	at org.hibernate.internal.FilterImpl.setParameter(FilterImpl.java:82)
	at org.grails.orm.hibernate.AbstractHibernateDatastore.enableMultiTenancyFilter(AbstractHibernateDatastore.java:342)
	at org.grails.orm.hibernate.AbstractHibernateDatastore$1.call(AbstractHibernateDatastore.java:368)
	at groovy.lang.Closure.call(Closure.java:430)
	at org.grails.orm.hibernate.GrailsHibernateTemplate$1.doInHibernate(GrailsHibernateTemplate.java:140)
	at org.grails.orm.hibernate.GrailsHibernateTemplate.doExecute(GrailsHibernateTemplate.java:243)
	at org.grails.orm.hibernate.GrailsHibernateTemplate.execute(GrailsHibernateTemplate.java:187)
	at org.grails.orm.hibernate.GrailsHibernateTemplate.executeWithNewSession(GrailsHibernateTemplate.java:137)
	at org.grails.orm.hibernate.AbstractHibernateDatastore.withNewSession(AbstractHibernateDatastore.java:322)
	at org.grails.orm.hibernate.AbstractHibernateDatastore.withNewSession(AbstractHibernateDatastore.java:331)
	at grails.gorm.multitenancy.Tenants.withTenantIdInternal_closure1(Tenants.groovy:182)
	at grails.gorm.multitenancy.Tenants$CurrentTenant.withTenant(Tenants.groovy:273)
	at grails.gorm.multitenancy.Tenants.withTenantIdInternal(Tenants.groovy:181)
	at grails.gorm.multitenancy.Tenants.withId(Tenants.groovy:157)

Because tenantId in Person is a Long you must make sure you supply a Long parameter to withId(1).

OkTestSpec.groovy:

Tenants.withId(1L) {
    new Person(firstName: "Groovy", lastName: "Grails").save(failOnError: true)
}

Tags: grails multi-tenancy 3.2.x


In Grails 2.5.1 don't name a request parameter "namespace"

Publicerad den 7 Sep 2015

Lessons learned the hard way...

When I upgraded a Grails 2.4.5 application to Grails 2.5.1 one group of events stopped working. No event listeners were found for the namespace and topic.

Reverting back to Grails 2.4.5 made the events work again.

I think the change that hit me was this commit. I had a request parameter called "namespace" that I used to dynamically set the namespace for the the events. But in Grails 2.5.1 "namespace" is a reserved parameter name for the controller namespace. So the value I sent in the request parameter was overwritten (actually it is removed) by Grails.

Renaming my request parameter from "namespace" to "ns" solved my problem. But I wonder if this is expected behaviour by Grails, removing "my" parameter like that?

Tags: grails 2.5.1 namespace upgrade problem


Scale and crop any image size to Full HD

Publicerad den 28 May 2015

If you need to scale images in various sizes and formats to same size (in this example to full HD) you can use ImageMagick and the following command:

convert in.jpg -resize "1920x1080^" -gravity center -crop 1920x1080+0+0 +repage -quality 50 out.jpg

Tags: web image convert


Use Perl to fix your old Grails controllers

Publicerad den 1 Mar 2012

Use Perl to replace Grails action references with deprecated format redirect(action: show) to redirect(action: 'show') and action closures to action methods

Today I finally took the time to upgrade my largest Grails 1.3.7 application to Grails 2.0.1

You can read about it here.

I started to develop this application during spring 2008 and I've been upgrading to all Grails versions between 1.1 and 1.3.7. This makes the code base somewhat "dynamic". Different code styles and implementations depending on Grails version, best practices at the time, and my skills level.

As a result, lots of old controller code has redirects that reference action closures directly using the action's property instead of the action name.

redirect(action:show, id:params.id)

instead of the recommended:

redirect(action:"show", id:params.id)

I did a "grep" on all my 70 controllers and found out that 50 of them (300+) lines had bad redirects!!!

No way I was going to change all those lines manually.

That's when my old Perl knowledge came to rescue!

Here's the script fixredirect.pl


#!/usr/bin/perl

while(<>) {
  if(/redirect(.+)action:\s*(\w+)/) {
    print "$`redirect$1action: \"$2\"$'";
  } else {
    print $_;
  }
}

 

$ cd grails-app/controllers/...
$ cp *Controller.groovy $HOME/i-am-a-chicken
$ perl -i redirectfix.pl *Controller.groovy

 
Done!
 
All redirect(action:prop) are now changed to redirect(action:"prop")
 
UPDATE!
 
The following script converts
def actionMethod = {arg1, arg2 ->
  // ...
}
to
def actionMethod(arg1, arg2) {
  // ...
}
 
But it's not tested in "production" so please use with caution.
 
#!/usr/bin/perl

while(<>) {
  if(/^\s*def\s+(\w+)\s*=\s*\{([^\-]*)\-\>\s*$/) {
    print "def $1($2) {\n";
  } elsif(/^\s*def\s+(\w+)\s*=\s*\{\s*$/) {
    print "def $1() {\n";
  } else {
    print $_;
  }
}
 
Good old Perl, thank you!

Tags: grails perl controller upgrade


Parse query string with Groovy

Publicerad den 31 Jan 2012

This little piece of Groovy code transforms a URL query string into a Map

One day I needed to parse a URL query string outside of a web container using only Groovy. I was surprised there was no standard was to do this and I could not find a small convenient library to use. So I rolled my own.

This is what I came up with:

// Define a complex query to test with.
def url = new URL("http://www.google.com/" +
  "?sclient=psy-ab&hl=sv&site=&source=hp&" +
  "q=parse+query+string+groovy&pbx=1&" +
  "oq=parse+query+string+groovy&aq=f&aqi=&aql=&gs_sm=e&" +
  "gs_upl=2640l7169l0l7417l25l14l0l11l11l0l102l616l13.1l25l0&" +
  "bav=on.2,or.r_gc.r_pw.r_cp.,cf.osb&fp=41e3a896b66de401&" +
  "biw=1368&bih=1010")

def map = url.query.split('&').inject([:]) {map, kv-> def (key, value) = kv.split('=').toList(); map[key] = value != null ? URLDecoder.decode(value) : null; map }

assert map.q == "parse query string groovy"

 

Works great! So let's add this method to the URL class

URL.metaClass.queryAsMap = {
  if(!delegate.query) return [:]

  delegate.query.split('&').inject([:]) {map, kv ->
    def (key, value) = kv.split('=').toList()
    if(value != null) {
      map[key] = URLDecoder.decode(value)
    }
    return map
  }
}

Tags: groovy