Thursday, September 29, 2016

CXF JAX-RS 2.0 - Perfect HTTP Spark Streaming Connector

Even the most conservative among us, the web services developers, will be better off admitting sooner rather than later that Big Data is not something that can be ignored, it has become a major technology in the software industry and will continue becoming even more 'influential' with the Internet of things wave coming in.

Where will it place your typical HTTP service which GETs some data for the users from some data store or accepts some POSTs with the new data ?

While I'm somewhat concerned seeing BigData consumers collecting the sources via a variety of custom optimized protocols and low-level transports like TCP, I firmly believe HTTP connectors should and will play a big role in connecting the WEB users with the Big Data processing chains.

HTTP, being so widely used, is a perfect frontend to the local networks where the nodes process the data, and while HTTP is synchronous for a typical interaction, JAX-RS 2.0 REST services can be quite smart. A variety of typical REST patterns can be employed, for example, a POST request handler with the data to be run through a BigData chain can let the application thread deal with it while respond to the user immediately, offering a link with a job id where the status can be monitored or the results returned from.  Or the handler can rely on the suspended HTTP invocations and start streaming the results back as soon they become available.

I have created a Spark Streaming demo showing some of the possible approaches. This demo is a work in progress and I will appreciate a feedback from the Spark experts on how the demo can be improved.

The demo relies completely on JAX-RS 2.0 AsyncResponse - the typical pattern is to resume it when some response data are available - and what is good it can be suspended multiple times to do a fine grained optimization of the way the service code returns the data. StreamingOutput is another piece - it allows writing the data back to the user as soon as they become available.  FYI, CXF ships a typed analog, called StreamingResponse, you can see how it is being indirectly used in this RxJava Observable test code.

But let me get back to the demo. It shows two types of Receivers in action.  This demo service shows how an HTTP InputStream can be converted to a List of Strings with a custom Receiver making them available to Spark. The service currently creates a streaming context per every request which I imagine may not be quite perfect but my tests showed the service performing quite well when the input set is parallelized - less than a sec for a half of MB PDF file.

Speaking of PDFs and other binary files. One of the service methods uses a beautiful Apache Tika Parser API which is wrapped in this CXF extension. These few lines of code is what it takes to have a service enabled for it to push the content of either PDF or OpenOffice documents to the Spark pipeline (I only added PDF and OpenOffice Tika Parser dependencies to the demo so far). I'm sure you are now starting wondering why Tika API is still not used in your JAX-RS services which parse PDF only with the PDF specific API :-)

I keep getting distracted. Back to the demo again. This demo service is a bit more closer to the real deployment scenario. It uses a default Spark Socket receiver - JAX-RS 2.0 service, being a good HTTP frontend, forwards the HTTP stream data to the internal Spark Streaming TCP server which processes the data and makes them available to a JAX-RS AsyncResponse handler which is also acting as a Socket server. The correlation between a given HTTP request and the Spark output data is achieved with a custom protocol extension. I can imagine it will be easier with an internal Kafka receiver which is something that the demo will be enhanced with to try later on.

In both cases, the demo streams the response data pieces back to the user as soon as they become available to the JAX-RS AsyncResponse handler.

Additionally, the demo shows a CXF JAX-RS Oneway extension in action. HTTP Client will get a 202 status back immediately while the service will continue with processing the request data.

I'm sure the demo will need more work but I also hope there's enough material there for you to start experimenting. Give it a try please and watch for the updates. I think it will be very interesting to see how this demo can also be written with Apache Beam API, check this blog entry for the good introduction.

Enjoy ! 

Progress In The JAX-RS 2.1 space

For those of you thinking what is going to happen to JAX-RS a good news is that JAX-RS 2.1 will live, surely it was the only possible outcome given the quality and the popularity of JAX-RS 2.0.

Check out this Java One 2016 Key Note, and I will continue standing by my assertion that even more is to come from JAX-RS.  


As far as Apache CXF is concerned, Andriy  has been working hard on a JAX-RS 2.1 branch where he implemented a 2.1 Server Sent Events API. And after Marek released the very first JAX-RS 2.1 API artifact to Central Andriy merged his work to the CXF 3.2.0 master branch.

It is very good because we can now start working toward releasing CXF 3.2.0 with this early JAX-RS 2.1 API to be implemented for the CXF users to experiment with - JAX-RS 2.0 users will be able to migrate to CXF 3.2.0 without changing anything in their services code.

The new JAX-RS 2.1 features (SSE, NIO and Reactive Invokers, with the early API improvements likely to happen during the coming specification work) are cool and it is worth taking the hats off to the engineering minds of the Jersey team for the top work they did.
  





Wednesday, September 28, 2016

[OT] Become Most Enigmatic Person in The Office - Discover CXF

Going the winding Apache CXF path is not that scary for a web services developer - some features may not be there just yet but you may discover something new instead, while helping driving the CXF forward along the way.

Have no fear, answer the call, help your team discover what Apache CXF really is. And become the most popular and enigmatic person in your office :-)

Tuesday, August 23, 2016

SwaggerUI in CXF or what Child's Play really means

We've had an extensive demonstration of how to enable Swagger UI for CXF endpoints returning Swagger documents for a while but the only 'problem' was that our demos only showed how to unpack a SwaggerUI module into a local folder with the help of a Maven plugin and make these unpacked resources available to browsers.
It was not immediately obvious to the users how to activate SwaggerUI and with the news coming from a SpringBoot land that apparently it is really easy over there to do it it was time to look at making it easier for CXF users.
So Aki, Andriy and myself talked and this is what CXF 3.1.7 users have to do:

1. Have Swagger2Feature activated to get Swagger JSON returned
2. Add a swagger-ui dependency  to the runtime classpath.
3. Access Swagger UI

For example, run a description_swagger2 demo. After starting a server go to the CXF Services page and you will see:


Click on the link and see a familiar Swagger UI page showing your endpoint's API.

Have you wondered what do some developers mean when they say it is a child's play to try whatever they have done ? You'll find it hard to find a better example of it after trying Swagger UI with CXF 3.1.7 :-)

Note in CXF 3.1.8-SNAPSHOT we have already fixed it to work for Blueprint endpoints in OSGI (with the help from Łukasz Dywicki).  SwaggerUI auto-linking code has also been improved to support some older browsers better.

Besides, CXF 3.1.8 will also offer a proper support for Swagger correctly representing multiple JAX-RS endpoints based on the fix contributed by Andriy and available in Swagger 1.5.10 or when API interface and implementations are available in separate (OSGI) bundles (Łukasz figured out how to make it work).

Before I finish let me return to the description_swagger2 demo. Add a cxf-rt-rs-service-description dependency to pom.xml. Start the server and check the services page:


Of course some users do and will continue working with XML-based services and WADL is the best language available around to describe such services. If you click on a WADL link you will see an XML document returned. WADLGenerator can be configured with an XSLT template reference and if you have a good template you can get UI as good as this Apache Syncope document.

Whatever your data representation preferences are, CXF will get you supported.

 




Monday, August 8, 2016

CXF Spring Boot Starters Unveiled

The very first check some new users may do these days, while evaluating your JAX-RS implementation, can be: how well is it integrated into SpringBoot ?

And the good news is that Apache CXF 3.1.7 users can start working with SpringBoot real fast.
 
We have left it somewhat late. It is hard to prioritize sometimes on various new requirements. And see some users moving away. In such cases the community support is paramount. And the Power of Open Source Collaboration came to the rescue once again when it was really needed.

I'd like to start with thanking James for providing an initial set of links to various SpringBoot documentation pages and reacting positively to the initial code we had. But you know yourself - sometimes we all value some little 'starters' - the initial code contributions :-)

And then we had a Spring Boot expert coming in and getting the process moving. Vedran Pavic helped me to create the auto-configuration and starter modules for JAX-RS and JAX-WS, patiently explained how his initial contribution works, how these modules have to be designed, and helped with the advice throughout the process. I felt like I passed some SpringBoot qualification exam once we were finished which let me continue enhancing the JAX-RS starter independently before CXF 3.1.7 was released.

CXF Spring Boot starters are now documented at this page which is also linked to from a Spring Boot README listing the community contributions.

If you are working with CXF JAX-RS then do check this section. See the demos and get excited about the ease with which you can enable JAX-RS endpoints, their Swagger API docs (and auto-link Swagger UI - the topic of the next post).

See how you can run your CXF WebClient or Proxy clients in Spring Boot, initialized if needed from the metadata found a Netflix Eureka. The demo code on the master uses a CXF CircuitBreakerFailoverFeature written by a legendary DevMind, a sound, simple and light-weight Apache Zest based implementation.
Not all users may realize how flexible CXF Failover Feature is. 

While the most effort went into a JAX-RS starter I'm sure we will add more support for JAX-WS users too.

We'll need to do a bit more work - link CXF statistics to the actuator endpoints, support scanning JAX-RS Applications and few other things.

If you prefer working with Spring Boot: be certain that a second to none support for running CXF services in Spring Boot will be there. Enjoy!



Wednesday, August 3, 2016

[OT] Reuse Or Reimplement ?

I said in one of my earlier posts I'd share some thoughts I've had during the years on re-using vs re-implementing while working on various CXF projects. Some of it may be a bit CXF specific but most of it might be interest to non-CXF developers too.


When the time comes to implement a new feature the immediate decision that needs to be taken is how you do it. In general it is always a good idea to re-use a quality 3rd party library that can help in realizing the new feature fast.


Consider a task of providing a UI interface to have Swagger JSON documents nicely presented. One can invest the time and write UI scripts and pages. Or one can download a well-known Swagger UI module.

Another example: one needs a collection sort algorithm implementation which will do faster than Java Collections code. One can invest a time and write a new library or look around and try an Apache or Google library.

In both cases re-using the existing solution will most likely be better and help deliver the higher-level, complete offering faster.

Things may get more complicated when one works on a project in a competitive space. For example, at some point there were 6 active JAX-RS Java implementation projects, with other non JAX-RS implementations such as the one offered by Spring adding up to the total number.

When you work on a project like that one a number of important decisions need to be made: how complete you'd like your project to be ? Is supporting HTTP verbs and reading and writing the data is all what is needed ? What sort of security support around the service you'd like to provide ? What other extensions should your project have ? How would you like your project be recognized - as a library or something bigger that offers all sort of relevant support for people writing the HTTP services ?

The higher the 'ambitions' of such a project the more likely the 're-implementing' becomes a viable option, nearly a necessity in some cases. In fact re-implementing is going all around at such projects.

I've been involved in a fair bit of re-implementation projects.

To start with we started implementing JAX-RS at a time when Jersey was already high. Why ? To have Apache CXF open to users with different preferences on how to do HTTP services. It was hard at times but it was really never simply because we wanted to prove we could do it.

The latest 're-implementation' was JOSE. Why ? I won't deny I was keen to work with the low-level security code closer, but overall, I wanted a CXF Security Story be more complete. Implementing it vs re-using the quality libraries I listed at the Wiki let us tune and re-work the implementation for it to be better integrated with the JAX-RS and Core security support so many times that it would be highly unlikely to happen if I were working with a 3rd party library.

I do not think re-implementing in an open way is not healthy. For example it has been acknowledged that having many JAX-RS implementations around helped to make JAX-RS more popular. Re-implementing may offer more options to users.

Or, reimplementing can prove a complete loss of time. Here are some basic 'guidelines' if you decide to try to re-implement in the Open Source:
- think not twice but many times before you try it
- if you feel the urge then do it, get the experience, make the mistakes, next time you will do the best choice
- never expect that once you re-implement something then everyone will stop using whatever they use and switch to what you have written - a lot of clever developers are working full time
- if you'd like others to use your project then you absolutely must love working with the users, don't even start if you think that it will be up to the Customer Support
- you need to have a support of your colleagues
- expect that the only 'remuneration' you will have is the non-stop work to keep the project constantly evolving

Yes, very often re-using may be the very best thing :-)

Enjoy, Happy Re-Using, Happy Re-Implementing :-)



 





Tuesday, June 21, 2016

Asynchronous JAX-RS Proxies in CXF

Dan had an idea the other day to get CXF JAX-RS proxies enhanced a bit for them to support the asynchronous calls. After all, HTTP centric JAX-RS 2.0 and CXF WebClient clients support such calls with AsyncInvoker.

So here is what we have started from. Simply register InvocationCallback with a proxy request context as shown in the examples and make the asynchronous call. The proxy method will return immediately and the callback will be notified in due time once the typed response is available. As the examples show one can register a single callback or a collection of callbacks bound to specific response types.

I suppose we can consider generating typed asynchronous proxy methods from the service descriptions such as WADL going forward.

This feature will be available in CXF 3.1.7. Give a try please, refresh your JAX-RS proxy code a bit, enjoy.