Technipelago Blog Stuff that we learned...
Logging HTTP traffic in Micronaut
If you want to see HTTP traffic between your Micronaut HTTP client and servers you can easily enable logging in logback.xml
src/main/resources/logback.xml
%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n
With this log level you will see both headers and body for HTTP requests and responses. See example below.
14:32:18.251 [nioEventLoopGroup-1-3] DEBUG i.m.http.client.DefaultHttpClient - Sending HTTP Request: POST /login 14:32:18.251 [nioEventLoopGroup-1-3] DEBUG i.m.http.client.DefaultHttpClient - Chosen Server: localhost(9005) 14:32:18.253 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - host: localhost:9005 14:32:18.253 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - connection: close 14:32:18.253 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - content-type: application/json 14:32:18.253 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - content-length: 79 14:32:18.253 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - Request Body 14:32:18.253 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - ---- 14:32:18.254 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - {"username":"user","password":"password","identity":"user","secret":"password"} 14:32:18.254 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - ---- 14:32:18.676 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - HTTP Client Response Received for Request: POST /login 14:32:18.677 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - Status Code: 200 OK 14:32:18.677 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - Date: Fri, 16 Nov 2018 14:32:18 GMT 14:32:18.677 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - content-type: application/json 14:32:18.677 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - content-length: 567 14:32:18.677 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - connection: close 14:32:18.677 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - Response Body 14:32:18.677 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - ---- 14:32:18.678 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - {"username":"user","roles":["ROLE_USER"],"access_token":"eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ1c2Vy...","expires_in":3600,"token_type":"Bearer"} 14:32:18.678 [nioEventLoopGroup-1-3] TRACE i.m.http.client.DefaultHttpClient - ----
Micronaut module name changes between RC2 and RC3
Once again Perl came to my rescue!
Many years ago I used Perl to batch update Grails source code.
When Micronaut changed module names between RC2 and RC3 I took the old Perl script from my blog and changed it to add micronaut- in front of all dependencies in build.gradle. I only had 15 build.gradle files to change so I could probably have done it manually, but there's many io.micronaut dependencies in each build file so a script is more convenient and eliminate the risk of typos.
In Micronaut RC2 and earlier a io.micronaut dependency could look like this:
compile "io.micronaut:inject"
In RC3 all module names was prefixed with micronaut-
compile "io.micronaut:micronaut-inject"
Here is the Perl script that change all io.micronaut:..... lines for you:
update-micronaut-module.pl
while(<>) { if(/(io\.micronaut[^:]*):(.+)/) { print "$`$1:micronaut-$2$'"; } else { print $_; } }
Make sure you checked in all build.gradle files to your VCS before continuing.
perl -i update-micronaut-module.pl */build.gradle
-i performs inplace editing, i.e overwrites the file after completion
WARNING: Don't runt the script twice, as it will add the micronaut- prefix each time you run it.
The road to Micronaut
I wanted to evaluate the new Micronaut framework and in the middle of September 2018 I made a bold decision...
During 3 months this summer I developed a proof-of-concept backoffice system with 10 microservices in Spring Cloud and a Vue.js front-end. It was a struggle to get all parts working. I used latest milestone relases of Spring components and that did impose some limitations and pain. I have 3-4 years experience with Spring Cloud but I had to cheat and create hacks and workarounds for things that didn't work so well. I'm sure all these issues can be fixed given enough time, education and when using release builds, but I had a demo deadline to meet.
I finished on time and the demo went great, but if you looked behind the scenes it was not pretty. :)
After the summer I wanted to evaluate the new Micronaut framework and in the middle of September I made a bold decision. Migrate all 10 services to Micronaut to get a feeling for the framework!
Micronaut 1.0.0 was not yet released when I started the migration project. I think I started on 1.0.0.M4, upgraded to RC1, RC2 and sometimes I was running on latest commits on the master branch. It was a bumpy road but I managed to solve all issues (the right way, not workarounds) and move on, much thanks to the amazing Micronaut team at OCI. They answered my questions on Gitter, I created issues on Github and they fixed them in no time. After 3 weeks all 10 services were up and running.
Things that I was struggling with in Spring Cloud, like Security, WebSockets, Apache Kafka, etc. was easy to get going in Micronaut. The most problematic part was service discovery with Eureka. But after Graeme Rocher fixed the last bug it all started to work like a dream.
Service Discovery, Security, WebSockets and Kafka are easy to configure (very little code) and testing HTTP endpoints with security enabled are fun again with Micronaut's declarative HTTP clients.
I'm absolutely sure that the Micronaut framework will have a GR8 future!
(a more detailed blog post will follow where I share the experiences gathered during this migration project)
Micronaut in a Spring Kafka environment
Spring Kafka encode message headers in a way that makes it incompatible with Micronaut (1.0.0.M4). This article provide one solution to the problem.
When Spring's Kafka implementation sends a message to Kafka it encode header values as JSON. So a simple string hello is written to the header as "hello". When Micronaut Kafka implementation recieves the message it will read the header value including the double quotes. So you header value is now ""hello"". This is probably not what you expected.
The other way around, when Micronaut sends a message to Kafka it just writes string headers as a sequence of bytes.When Spring Kafka decodes the header it will get a byte array instead of a String. And if the implementation expects a String and do messageHeaders.get("headername", String.class) an exception will be thrown.
If you only allow string values in your Kafka headers there is an easy solution on the Spring side to make it compatible with Micronaut (and other non-spring services). But this solution alters the message format for all messages in you cluster so it may not be an option in your environment.
The proposed solution is to add a custom KafkaHeaderMapper in each Spring service that read/wrie messages to Kafka.
@Bean public KafkaHeaderMapper myKafkaHeaderMapper() { return new KafkaHeaderMapper() { @Override public void fromHeaders(MessageHeaders headers, Headers target) { for (Map.Entry<String, Object> entry : headers.entrySet()) { target.add(entry.getKey(), entry.getValue().toString().getBytes()); } } @Override public void toHeaders(Headers source, Map<String, Object> target) { for (org.apache.kafka.common.header.Header header : source) { target.put(header.key(), new String(header.value())); } } }; }
Creating the bean is not enough, you must also add a reference to the bean name in application.yml.
spring: cloud: stream: kafka: binder: headerMapperBeanName: myKafkaHeaderMapper
Now the Spring service will expect header values to be just plain byte arrays and it will map to/from String accordingly.
Original issue https://github.com/micronaut-projects/micronaut-core/issues/605
Custom error response for Spring REST endpoints
It's easy to customize the error response from REST endpoints in Spring Boot. This blog post explains how to add a custom error code to the response.
The default error response from a Spring Boot REST endpoint looks something like this:
{ "error": "Not Found", "exception": "com.mycompany.rest.MyException", "message": "A record with value [foo] was not found", "path": "/foo", "status": 404, "timestamp": 1507117452642 }
If we want to include custom information into the error response it's easy. Just add an ErrorAttributes bean to the spring context.
The following code snippet checks if the thrown exception is a custom exception MyException. This exception type contains a code attribute that holds a unique error code. We want to include that code as a specific key in the error response.
@SpringBootApplication @EnableResourceServer public class MyApplication { public static void main(String[] args) { SpringApplication.run(MyApplication.class, args); } @Bean public ErrorAttributes errorAttributes() { return new DefaultErrorAttributes() { @Override public MapgetErrorAttributes(RequestAttributes requestAttributes, boolean includeStackTrace) { final Map<String, Object> errorAttributes = super.getErrorAttributes(requestAttributes, includeStackTrace); // Customize the default entries in errorAttributes by adding the error code. Throwable exception = this.getError(requestAttributes); if (exception instanceof MyException) { errorAttributes.put("code", ((MyException) exception).getCode()); } return errorAttributes; } }; } }
If a controller method throws MyException the response now looks like this:
{ "code": "12345", "error": "Not Found", "exception": "com.mycompany.rest.MyException", "message": "A record with value [foo] was not found", "path": "/foo", "status": 404, "timestamp": 1507117452642 }
Grails 3.2.0 - java.lang.IllegalArgumentException: Incorrect type for parameter [tenantId]
With Grails 3.2.0 native multi-tenancy has finally arrived in Grails core. This makes Grails a powerful framework for multi-tenant web applications. But these advanced features comes with a price. New kind of problems surface and you most know what you're doing more than ever.
Tested with Grails 3.2.0.RC1
Consider the following code:
Person.groovy: import grails.gorm.MultiTenant class Person implements MultiTenant{ Long tenantId String firstName String lastName def beforeValidate() { if(tenantId == null) { tenantId = Tenants.currentId() } } }
FailTestSpec.groovy: Tenants.withId(1) { new Person(firstName: "Groovy", lastName: "Grails").save(failOnError: true) }
This looks ok, doesn't it?
Well, this is what you get:
java.lang.IllegalArgumentException: Incorrect type for parameter [tenantId] at org.hibernate.internal.FilterImpl.setParameter(FilterImpl.java:82) at org.grails.orm.hibernate.AbstractHibernateDatastore.enableMultiTenancyFilter(AbstractHibernateDatastore.java:342) at org.grails.orm.hibernate.AbstractHibernateDatastore$1.call(AbstractHibernateDatastore.java:368) at groovy.lang.Closure.call(Closure.java:430) at org.grails.orm.hibernate.GrailsHibernateTemplate$1.doInHibernate(GrailsHibernateTemplate.java:140) at org.grails.orm.hibernate.GrailsHibernateTemplate.doExecute(GrailsHibernateTemplate.java:243) at org.grails.orm.hibernate.GrailsHibernateTemplate.execute(GrailsHibernateTemplate.java:187) at org.grails.orm.hibernate.GrailsHibernateTemplate.executeWithNewSession(GrailsHibernateTemplate.java:137) at org.grails.orm.hibernate.AbstractHibernateDatastore.withNewSession(AbstractHibernateDatastore.java:322) at org.grails.orm.hibernate.AbstractHibernateDatastore.withNewSession(AbstractHibernateDatastore.java:331) at grails.gorm.multitenancy.Tenants.withTenantIdInternal_closure1(Tenants.groovy:182) at grails.gorm.multitenancy.Tenants$CurrentTenant.withTenant(Tenants.groovy:273) at grails.gorm.multitenancy.Tenants.withTenantIdInternal(Tenants.groovy:181) at grails.gorm.multitenancy.Tenants.withId(Tenants.groovy:157)
Because tenantId in Person is a Long you must make sure you supply a Long parameter to withId(1).
OkTestSpec.groovy: Tenants.withId(1L) { new Person(firstName: "Groovy", lastName: "Grails").save(failOnError: true) }
In Grails 2.5.1 don't name a request parameter "namespace"
Lessons learned the hard way...
When I upgraded a Grails 2.4.5 application to Grails 2.5.1 one group of events stopped working. No event listeners were found for the namespace and topic.
Reverting back to Grails 2.4.5 made the events work again.
I think the change that hit me was this commit. I had a request parameter called "namespace" that I used to dynamically set the namespace for the the events. But in Grails 2.5.1 "namespace" is a reserved parameter name for the controller namespace. So the value I sent in the request parameter was overwritten (actually it is removed) by Grails.
Renaming my request parameter from "namespace" to "ns" solved my problem. But I wonder if this is expected behaviour by Grails, removing "my" parameter like that?
Scale and crop any image size to Full HD
If you need to scale images in various sizes and formats to same size (in this example to full HD) you can use ImageMagick and the following command:
convert in.jpg -resize "1920x1080^" -gravity center -crop 1920x1080+0+0 +repage -quality 50 out.jpg
Use Perl to fix your old Grails controllers
Use Perl to replace Grails action references with deprecated format redirect(action: show) to redirect(action: 'show') and action closures to action methods
Today I finally took the time to upgrade my largest Grails 1.3.7 application to Grails 2.0.1
I started to develop this application during spring 2008 and I've been upgrading to all Grails versions between 1.1 and 1.3.7. This makes the code base somewhat "dynamic". Different code styles and implementations depending on Grails version, best practices at the time, and my skills level.
As a result, lots of old controller code has redirects that reference action closures directly using the action's property instead of the action name.
redirect(action:show, id:params.id)
instead of the recommended:
redirect(action:"show", id:params.id)
I did a "grep" on all my 70 controllers and found out that 50 of them (300+) lines had bad redirects!!!
No way I was going to change all those lines manually.
That's when my old Perl knowledge came to rescue!
Here's the script fixredirect.pl
#!/usr/bin/perl while(<>) { if(/redirect(.+)action:\s*(\w+)/) { print "$`redirect$1action: \"$2\"$'"; } else { print $_; } }
$ cd grails-app/controllers/... $ cp *Controller.groovy $HOME/i-am-a-chicken $ perl -i redirectfix.pl *Controller.groovy
def actionMethod = {arg1, arg2 -> // ... }
def actionMethod(arg1, arg2) { // ... }
#!/usr/bin/perl while(<>) { if(/^\s*def\s+(\w+)\s*=\s*\{([^\-]*)\-\>\s*$/) { print "def $1($2) {\n"; } elsif(/^\s*def\s+(\w+)\s*=\s*\{\s*$/) { print "def $1() {\n"; } else { print $_; } }
Parse query string with Groovy
This little piece of Groovy code transforms a URL query string into a Map
One day I needed to parse a URL query string outside of a web container using only Groovy. I was surprised there was no standard was to do this and I could not find a small convenient library to use. So I rolled my own.
This is what I came up with:
// Define a complex query to test with. def url = new URL("http://www.google.com/" + "?sclient=psy-ab&hl=sv&site=&source=hp&" + "q=parse+query+string+groovy&pbx=1&" + "oq=parse+query+string+groovy&aq=f&aqi=&aql=&gs_sm=e&" + "gs_upl=2640l7169l0l7417l25l14l0l11l11l0l102l616l13.1l25l0&" + "bav=on.2,or.r_gc.r_pw.r_cp.,cf.osb&fp=41e3a896b66de401&" + "biw=1368&bih=1010") def map = url.query.split('&').inject([:]) {map, kv-> def (key, value) = kv.split('=').toList(); map[key] = value != null ? URLDecoder.decode(value) : null; map } assert map.q == "parse query string groovy"
Works great! So let's add this method to the URL class
URL.metaClass.queryAsMap = { if(!delegate.query) return [:] delegate.query.split('&').inject([:]) {map, kv -> def (key, value) = kv.split('=').toList() if(value != null) { map[key] = URLDecoder.decode(value) } return map } }