Feb 16 2010

The denormalizer in CQRS

Category: Administrator @ 18:06

The “denormalizer” is a component responsible for mapping data from the domain model to a denormalized table structure at the query side (an over can be found here). Usually the denormalizer component runs as a windows service at the server hosting the query side. Its only responsibility is to move data from the command side to the query side, and function as a router for messages from the write side. You might not want the same data on all the read servers.

Database tables matching view models

On the query side, all the tables match a view model in the GUI. Yes, this means you will end up with some duplicated data in the database tables, but that’s ok. Hard disk space is cheap.

Let’s look on how the denormalizer fits into the architecture.

image

When the domain model have done its stuff, an event is raised and a message is sent through a “pipeline” to the denormalizer on the query side. The “pipeline” can be a queue or something else that is fit to move the message in a safe way. I have used NServicebus with a lot of success, but there are other ESBs or other ways to do this.

Easy programming model

When the denormalizer receives the message, it spreads the data into the tables that needs to be updated . The tables have a one-to-one relationship with the view models. A GUI view can have multiple view models. What this gives you is a very easy programming model on the query side. All you have to think about concerning the denormalizer, is to insert/update the data to the correct tables. From the client, you look at the data as reporting data; meaning you can’t update or insert data on the query side.

NOSQL

All though I’m referring to a relational database at the query side in this post, it doesn’t have to be a relational database. You can use a non relational database if you want to. I would recommend MongoDB over a object database like DB4o, but that’s just my taste at the moment. You can even build your own key-value store because in most GUI views you will just fetch data based on an id.

Summary

To sum it up; the denormalizer routs the messages from the command side in to the correct tables on the query side.

Tags: ,

Currently rated 4.3 by 3 people

  • Currently 4.333333/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5

Jan 7 2010

Five things to think about when working with messaging solutions

Category: DDD | Patterns | Messagesfossmo @ 16:21

I’m trying to gather some of my thoughts around messages and that resulted in these 5 points:

1) A message can be created with any technology. It can be JSON, XML, Binary or a custom format. If you want to use the message in different technologies (like Java and .NET), it might be smart to go for a format supported by both technologies. Well, if you choose a technology like XML, it’s not given that you can exchange messages. XML is just a way to format your message, it doesn’t say anything about the content.

2) A message consists of two basic parts; a header and a body.
The header describes the data being transmitted and the body holds the data being transmitted.

3) The tree most important message types are (as I see it); document messages, command messages, event messages. You can read more about this messages in the book Enterprise Integration Patterns: Designing, Building, and Deploring Messaging Solutions (I’m linking to the kindle edition. I love my Kindle :-) ).

A bit more about this message types: A command message is a regular message that contains a command. It uses the command pattern to, for example, encapsulate the request and send it to another system. A document message is used to transferee a data structure between applications and a event message is used to notify another system about a change; for example a price change.

4) When working with messages, you need a way to handle them. On the .NET platform there are several open source choices; NServicebus, Rhino Servicebus, Simple Servicebus and MassTransitt (it’s probably more solutions out there). I think all of them uses MSMQ when communicating the messages, but there are other ways too. There are probably some commercial products out there too, but I haven’t tried to look them up at the moment. For the record; you don’t need a enterprise service bus handle messages. A message can be used to communicate internal in an application, for example between bounded contexts.

5) Messages are immutable. You can look at them as value objects. Greg Young puts it like this; Mutable messages are an anti-pattern. You can read more about his at Greg Youngs blog.

Tags: , ,

Be the first to rate this post

  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5

Dec 8 2009

Update to the CQRS figure

Category: DDD | CQRSfossmo @ 16:33

I did an update to the figure showing how I have implement the CQRS architecture.  As you can see from the figure, I have added a “message-pipeline” from the application service to the denormalizer. The reason is to show that all the updates to the read databases goes through this “pipeline”. The messages will for example be added to a queue or bus. The read databases then will pick up the new messages from the queue or bus, and update itself. I have left the synchronization arrow in the figure because it is used to initialize the read databases based on the write database. I have also done a update on how I have named the different layers and tires in the figure.

image

Tags: ,

Currently rated 5.0 by 1 people

  • Currently 5/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5

Nov 23 2009

Command and Query Responsibility Segregation (CQRS)

Category: DDD | DND | CQRSfossmo @ 16:29

It’s almost one year since I read the “blue book”. It changed my view on how I write software and how I build software architecture. During my time as a software developer, I have tried many different approaches on building software. A lot of the approaches includes an anemic domain model. It’s nothing wrong with building an anemic domain model, but for applications with a bit more complex business logic, it might not be the best choice. You can end up with a lot of “spaghetti code” with high coupling between the different parts of the code. Anemic domain models have their business logic spread “all over” the code and you often have to do updates multiple places in your code if a business rule is changed. And you want to avoid that. Keep this in mind when you code:

“Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live” – M. Golding

A classis rich domain model holds all the business logic inside the model and most of the objects are connected to eachother. It strives to be a model that solves the domain logic for the business in one perfect model. And this is often where this way of writing software fails. You get a big model that finally gets very unwieldy. You should rather think bounded context, but that’s not the topic for this post.

Split the model

pie Instead of trying to solve everything in one model; split it into smaller parts and work with each part on its own. Create aggregates that naturally fits together. An example can be a car and a customer. In this example, both are aggregate roots. You should consider to not model both objects and their aggregates into one model. Think of them as small separated models inside the model. This will make the model much easier to handle when persisting and loading into memory.

Greg Young had a great example when he was explaining aggregate roots and how they should work. If a teacher is asked by the headmaster for a overview of his students, he will not bring all the students to the the headmasters office, but he will bring a list with the student information the headmaster wants. For the most part, there are no good arguments for putting everything in one big connected model.

The problem with relational databases

Nowadays many use ORM frameworks to persist data from the domain model to the database, and they use a relational database as the database (are there others? ;-)). A problem with the relation databases is that you have to do a compromise on speed regarding reading and writing data. If you normalize the database, it will be better at inserting, updating and deleting data, but not reading data. The problem are indexes. These buggers will in most cases slow down your select operations but help insert, delete and update operations.

 Update: I have received some comments on this statement. What I was trying to say is: If you add indexes to improve update/delete operations, it might affect the selects you do against the database. You have to do a compromise; either you read faster or write faster.

If you decide to make the database better at reading data (selecting), one option is to de-normalize the database. A de-normalized database means tables with repetitive data and possibly tables with a number of columns from hell.

As developers we often don’t give much thought to this question; how to scale the solution? I think we should keep this thought in the front of your head when you create a solution that may have to serve a lot of users. I’m not saying you should try to predict the future, but with some simple measures from the start, you do not have to think much of it later.

Command and Query Responsibility Segregation

Most applications reads data more frequently than they write data. Based on that statement, it would be a good idea to make a solution where easily you can add more databases to read from, right? So what if we set up a database dedicated just for reading? Even better; what if we design the database in a way so it’s faster to read from it? If you design your applications based on the patterns described in CQRS architecture, you will have a solution that is scalable and fast at reading data. Command Query Separation (CQS) is a pattern that was first thought of by Bertrand Meyer. He described it on the object level. Later this pattern has been lifted from the lower levels and has been used in the higher architectural level. I think it was Udi Dahan who first started talking about this architectural principle.

In this post I describe an architecture that is quite close to the one Udi is talking about. There are several implementations of this pattern, and one of them is described by Greg Young. Greg Young's flavor of the pattern describes CQRS and Event Sourcing. I won’t go into this flavor in this post. If you want to learn more about this flavor, Google for it.

A quick walkthrough of the CQRS architecture

A graphical overview of the architecture is shown in the figure below. Lets do a quick walkthrough of the figure. A user opens an application and the first screen is loaded. The data is fetched from the Query side. In this example it goes through a WCF service and uses, lets say NHibernate, to read data from the database and into a DTO that returns the data to the GUI. The DTO is tailored to fit the screen the user is viewing. The query database is often de-normalized to improve reading. The user may brows through different screens, and the process is the same. A DTO tailored for the users screen is returned from the database. Eventually, the user wants to change the data in one of the screens. What then happens is that a command message is created based on the data that have changed in the view and sent down the left side of the figure; the command side. The command message is sent into the domain model to get validated against the business rules. A error message is sent back to the client if some of the business rules fails (how this is done can differ). If the message goes through the domain model without errors, it’s persisted to the write database and synchronized with the read database(s). The command side should never return other data than the error message. If you follow this rule, the command side will only be behavioral. This makes logging of what happens in the domain model very easy, and it’s quite easy to track what the user wanted to do, not only what the result of his actions were. The approach I’m describing in this post is a bit different than Greg Young describes. His solution promotes no database on the command side; a database is only needed for reporting. This is a great approach, but it makes things a bit more complex and it requires a green field project(?). The approach I’m describing can be used with a brown field application without much change to the existing architecture. In most cases you only have to add a query-side to the “old” architecture.

Why add query side DTOs?

In a traditional domain model, you will create the objects for solving domain logic, not for viewing. They have behavior rather than shape. To make domain objects more viewable, many developers use mapping between the domain model and a DTOs tailored for being displayed. This results in a lot of mapping between objects for the developer and domain models who exposes getters and setters (To learn why this is bad, do a Google search on; getters and setters are evil).

If you are using a ORM like NHibernate, you have to add getters to the domain model (and make the properties and methods virtual if you want to use lazy loading), and I think this is ok. The model is still protected against unwanted changes that can make it invalid. Every change have to happen through its command methods.

Summary

What you get from this architecture is; scalability on the read side of the application (where you normally get most load), the possibility to log all happenings in the domain model (by tracking the commands and events in the domain model), and a domain model that only have to worry about writing data, not be a transporter of data from the database to the GUI. I guarantee that this alone will make things a lot easier for you. And one final thing; creating objects (DTOs) based on the view helps us avoid a lot of mapping and a lot of requests against the database to fill the view with data. I will go through the parts of the architecture in more details in future posts.

Update: I have done a update to this figure in a new post.

image

Tags: ,

Currently rated 3.2 by 6 people

  • Currently 3.166667/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5

Jul 2 2009

A great book to get you started with NHibernate

Category: Administrator @ 15:13

Lately I have tried to learn NHibernate. NHibernate is a object-relational-mapper. It maps data from relational databases to POCO-objects. So, what is useful about that, you may ask? Well, for a starter, it’s very handy when creating a rich domain model. If you read this blog regularly, you may know that I’m also in the process of learning Domain Driven Design.

Domain Driven Design and NHibernate

The idea behind DDD is that you focus on the domain model instead of focusing on the database or GUI when trying to capture the business value from a domain expert. If you follow the guidelines described in the books written by gurus on the topic, you will end up with a domain model that is loosely coupled from the other parts of the system you are creating. That means you don’t have a coupling against the GUI, the database or any other parts of the system. But, you still need to persist the data from the domain model to the database in some way, and this is where NHibernate comes in to play. By setting up a mapping, described in a file, from the domain model to the database you can make NHibernate do the mapping for you. How cool is that? I guess you knew that already, but the purpose of this post is not to tell you how cool NHibernate is, but to guide you in the right direction of how to start learning NHibernate.

How I learned it

image

My first approach to learning NHibernate was to look at the excellent screen casts made by Stephen A. Bohlen called Summer of NHibernate. I learned a lot from watching these screen casts, but I still felt like something were missing. It’s like when you go to a conference, at the end of the day you can’t sit down in front of the PC and start to crank out code based on what you learn that day. But, it gives you a starting point. You often get information of how to investigate the topic further. This is what Summer of NHibernate did for me and it pointed me in the direction of a book called NHibernate in action.

Beside giving you a good start on setting up and using NHibernate, it also explains how to write real-world domain models. It tells you how to set up pretty complex associations between entities and why it’s important to understand why a domain model should be persistence ignorant and so on and so fourth. If you want to have a look at the index of the book, it can be found at this link.

If you want to learn NHibernate you definitely should look at this book.

Links

Links to NHibernate resources I have used and use:

- Summer of NHibernate (screen casts)
- Dimecasts.org have some screen casts on the topic (max 10 minutes long)
- NHibernate in action (excellent book on the topic)
- NHibernate.org (You can download the binaries from this site and you find documentation)

Happy learning!

Tags:

Currently rated 5.0 by 1 people

  • Currently 5/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5

May 19 2009

Speaking at NNUG in Trondheim 28.05.2009

Category: NNUG | DDD | Presentation | Speakerfossmo @ 14:16

image At the next Norwegian .NET User Group meeting I’m going to talk about my experience with Domain Driven Design. I hope we can get a discussion around the topic after the talk. I’m really exited and looking forward to the meeting. Sign up for the meeting here.

Tags: ,

Be the first to rate this post

  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5