The Grok story
The Story of Grok.
The Grok Story
What is Grok?
Grok is a web application framework for Python developers. It is aimed at both beginners and very experienced web developers. Grok has an emphasis on agile development.
Currently Grok is at version 1.0! Since Grok is built using Zope 3, which has been in development since 2001, we think you'll find Grok to be very stable. However, some areas of Grok are still more challenging to learn, but we are working on it!
Learn why Grok is an interesting web framework.
You will likely have heard about many different web frameworks for Python as well as other languages. Why you should you consider Grok?
- Grok offers a lot of building blocks for your web application.
- Grok is informed by a lot of hard-earned wisdom.
Grok accomplishes this by using at its core the Zope Toolkit (ZTK), an advanced object-oriented set of libraries intended for reuse by web frameworks. While Grok uses the Zope Toolkit, and benefits a lot from it, you can get started with Grok without any special knowledge of the ZTK.
Some aspects of Grok development require more detailed knowledge of the ZTK libraries (authentication, for example) but it is our goal to make Grok a more approachable, easier to learn framework over time. If you find yourself bonking when trying to accomplish something, please let us know where you are struggling.
Grok doesn't require you to edit cryptic configuration files. Instead you just program in Python and create HTML templates. Beyond this, Grok also offers a wide range of built-in features at your fingertips, from automated form generation to an object database. This way, Grok gives you both power and quick satisfaction during development. Grok is fun.
Through the ZTK, Grok offers a very wide range of building blocks for your web application. What's more, ZTK libraries and components are typically rock-solid due to extensive unit-testing and well-specified API documentation.
Grok, thanks to the ZTK, is informed by hard-earned wisdom. The core of the ZTK has been under continuous development since 2001. The ZTK's design in turn is based on experience with the Zope 2 platform, which has been under continuous development (starting with Bobo, Principia and then Zope 1) since 1997. The Zope community is supported by an independent foundation, the Zope Foundation.
The Zope community has been around for a while. We've built a lot and learned a lot. We are in this for the long run.
Successful web applications aren't built for a day - such an application will need to be maintained, extended, evolved, over a period of many years. Zope developers really know this. Grok, through the ZTK, offers an architecture that enables your application to grow over time.
Check out What Grok code looks like.
import grok class HelloWorld(grok.Application, grok.Model): pass class Index(grok.View): pass index = grok.PageTemplate(""" <html><body> <p>ME GROK HELLO WORLD!</p> </body></html> """")
- Herd of Mammoths: a very simple application that only goes a bit beyond "hello world".
- Grokstar: a simple blog application written with Grok.
- Grok Wiki: a simple wiki application written with Grok.
- Paleosoft.org: a collection of sample apps written in Grok/Zope3 by Luciano Ramalho as part of the Google Summer of Code.
- GHOP - Introduction to Grok is an introductory screencast produced as part of the Google Highly Open Participation contest.
- Simple ToDo application is a tutorial-style introduction to Grok by Philip von Weitershausen.
- Introduction to Grok is a talk given by Philipp von Weitershausen to a group of Zope developers.
Grok is informed by hard-earned wisdom and offers a lot of building blocks for your application. So what does Grok offer that we think is special?*
Grok is an integrated megaframework. That is, it features replaceable best-of-breed components (megaframework) while still aims to give you an integrated feel. The parts that Grok is constructed from (mostly, the Zope Toolkit libraries) follow common style guidelines.
The Zope community that Grok is a part of has had years of experience building powerful, pluggable Python-based web applications. The community goes back to 1998 or even longer ago, if you count Zope's predecessors bobo and Principia. We know that web applications need to grow and change over time. We aren't just aware of it; we know it through hard-fought experience maintaining large systems built with Zope. We also know that the same web application is often deployed in different contexts and organizations. We want to build applications that can be extended, and components we can use again elsewhere.
Grok is special because it offers a lot of components, and more, a way to glue components together. We believe of these components go beyond to what the web-framework competition has to offer.
The Zope community, for years now, has focused on building components. Components are powerful, pluggable building blocks you can use to build your application.
When you use Grok to build an application, all these components are available to you. Here are some components to help you to do a lot of things:
- auto-generate forms from schemas, with a wide selection of widgets. (zope.formlib, built into Grok. z3c.form is another approach)
- create vocabularies (for drop-down lists in forms, and other lists in applications) (zc.sourcefactory).
- application skinning (built into Grok).
- create part-of-page views, per context object, per view, per area on page, per skin (viewlets). (built into Grok)
- display data-driven HTML tables (zc.table).
- integrate with ORMs (Storm, SQLAlchemy).
- sessions (built into Grok).
- authentication: basic auth, session-based auth, pluggable user databases supporting LDAP, relational databases, etc. (pluggable authentication architecture built into Grok).
- authorization: users, groups, per-user/per-object permissions and roles. (buit into Grok).
- REST, JSON, XMLRPC (built into Grok)
- WebDAV support. (z3c.dav)
- declarative Ajax using KSS.
- i18n, localization of things like date/time, country names, etc. (built into Grok)
- freezing objects. (zc.freeze)
- sending, queuing mail. (zope.sendmail)
- workflow. (hurry.workflow, zope.wfmc)
- XML-export and import of schema-driven content. (z3c.schema2xml)
- object indexing and search. (built into Grok, and hurry.query)
- Lucene, Xapian indexing integration. (z3c.indexing.xapian, z3ext.lucene)
- relation indexing (zc.relation, z3c.relationfield).
- making objects appear in multiple places at once in the traversal hierarchy, including generating correct URLs for them and the URLs of sub-objects (zc.shortcut).
- versioning objects in a version control system (zc.vault).
- caching, support for protocols like ICP (zope.app.cache, zope.cachedescriptors, lovely.memcached, lovely.responsecache, lovely.viewcache, zc.icp)
- unit testing, doc testing, functional testing, fake browser doc testing, driving Selenium from browser doc testing, driving a real browser from browser doc testing, and so on. (built into Grok, zc.testbrowser)
- application flash messages (z3c.flashmessage, built into Grok).
- an extensible application buildout system that can install a lot more than just Python packages (zc.buildout).
And that's just a grab-bag list. These components offer an integrated experience: they are packaged as eggs, define explicit APIs (using interfaces), and are typically doctested (frequently extensively so).
All these Zope Toolkit components, and many more, can be used with Grok, as Grok uses the same underlying architecture. But with Grok, we consider the existence of a component the half-way-point; now that it's there, we should make it easier. We focus on the developer experience. We do this in three ways:
- Improve the documentation for the use of individual components. We want to lower the entry barrier by providing straightforward documentation.
- Continue to work on making individual components more easy to use. By integrating a particular component with Grok and providing sensible defaults, we can make that component more accessible to developers. We are continuously working on this - a task that likely never will end given the creativity and productivity of the Zope Toolkit component developers.
- Configuration (glueing) of components, for instance glueing a view to a content object, has been harder than it should be with Zope 3. With Grok we introduce a much easier, uniform mechanism to glue things together, in Python code. Since all Zope Toolkit components make use of the same component system, all these components become easier to configure if you use Grok.
We don't claim we can make all complexity go away in software development. If you are going to do hard stuff, you are going to have to try to understand the various Zope Toolkit components you want to do hard stuff with, or build a new one yourself. This is an inescapable fact of software development. What we try to do is push the boundaries of what is actually hard. Zope Toolkit components do this by offering new features and building abstraction layers. With Grok we add easier assembly of components, and pre-assembly.
Web applications, if they want to stay relevant over time, need to be extensible with new behavior, and it should be possible to override aspects of existing behavior as well.
When developing software, we like keeping the core of applications clean, and we like modularity. This gives us flexibility in the way we deploy applications. We want to be able to extend, or plug into, a core application from another package, without having to change the original application itself.
An example: take an application that provides some way for users to create and manage documents, like a blog or a CMS. In this application, a document is represented by a document content object. Now imagine that this application does what you need for one thing. This probably sounds familiar: this would be perfect if only it had this extra feature. Let's say what you want is a commenting feature. Let's also say that other people want to deploy the same application without this feature. And that some other people have an entirely different approach to commenting and don't like the way you want it to work. These requirements make it desirable that the application itself doesn't provide a commenting feature directly. It should however be possible to plug one in.
Grok provides patterns that allow you to extend the application from the outside. If you want to add a new commenting view (a new web page) to every document object in the application, the core application doesn't even need any preparation. This is just possible straight away with the way Grok works - just hook the view up to the document class from an extension. What want to take a special action when a document object gets added, like initializing the comment data structure? No need to modify the core application either: you can hook into the add event that the core application sends when an object is added to a container. How do you store the comments? Grok offers an annotation framework that allows you to cleanly add new data to an existing object, without the original object having to know about it. You can add arbitrarily complex data structures, as Grok comes with an object database out of the box. If you want to extend existing views, the core application does need some minimal preparation: it has to open a slot in its document template. Arbitrary extensions can then plug new sub-views into it, using viewlets.
We call these patterns the component architecture. The component architecture that Grok uses was created to allow developers to do extensibility in an uniform way. Instead of building endless ad-hoc pluggability and extension points into an application, we have one general, powerful Python library, zope.component, that helps us build these pluggability points. zope.component is in its foundations just a registry that helps you plug objects together, but it's a very powerful registry which allows some advanced patterns of glueing objects together. The Zope Toolkit libraries that Grok builds on use the component architecture, and thus you are dealing with a uniform, integrated story for pluggability when you work with any of these components.
Can other web frameworks do what Grok can do, with its Zope technologies? Haven't they caught up with Zope by now? What makes Grok special?
Of course other web frameworks can do what you can do with Grok: build web applications. We have been doing it for a while in the Python community, but other Python web frameworks have certainly come a long way since the development of Zope started. Other web frameworks now also do things like model/view separation, and automatic form generation. We also don't deny that other web frameworks have started doing things that Zope didn't do yet: object relational mapping (ORM) is an example of this, though even that is debatable - our community did do some early pioneering in that area with things like ZPatterns, back in the year 2000 or so.
In other ways, Grok still offers many features that other systems don't offer. If we didn't believe we were doing some things better, we wouldn't be working with Grok!
The rest of the Python web world still typically doesn't use object databases. Some people only dream about an object database as some form of "future magic" that will finally allow them to eliminate the impedance mismatch between object oriented programming and a relational database. Meanwhile in the Zope community we are so used to an object database we know about all the drawbacks too. We therefore also offer ORM integration in addition to an object database. The combination of both opens up new possibilities.
The highly advanced CMSes built with Zope offer some evidence that the use of an object database helps developers to combine objects and components into new higher-level frameworks and applications. This is perhaps a bit more easy than with relational database systems, where everybody needs to coordinate on database schemas.
We believe Zope is also ahead in providing an advanced permission-based security system. Zope is ahead of the curve in working with an extensible buildout system, though zc.buildout is also being adopted by others now.
Zope's strongest area is undoubtedly the component architecture, which provides a uniform system for extensibility, and the catalog of powerful components that we've built. We've seen a vast explosion of components over time - evidence that our component-driven approach is working.
We are forward-thinking about making Zope technologies reusable in other contexts. We also actively look to reuse in Grok what web frameworks developed, such as WSGI. We are all about component creation, exchange and reuse! We see the Python web framework world as an ecosystem where there is more cooperation between frameworks than competition.
Grok is built on the ZTK.
Grok stands on a giant's shoulder. That giant is the Zope Toolkit (ZTK).
The ZTK is a collection of libraries and mini-frameworks. Together they offer an advanced object oriented approach towards web development. The ZTK features a large amount of API documentation and aims for reliability. It has a very large automatic test coverage (many thousands of tests). It has a large set of core features, and sports an enormous range of plug-in components.
The Grok developers use the ZTK as the foundation of Grok. Grok turns it into a complete web framework, and also puts a twist on some of the approaches in the ZTK to make them easier to use. This is because the power offered by the ZTK can raise the entry barrier for developers. In addition, the ZTK's emphasis on explicit configuration and specification of interfaces can slow down development.
Grok aims to tackle these problems. Grok builds on the ZTK to create a web framework that's easy to learn, and agile to work with, while retaining the underlying power and compatibility with other systems that also use the ZTK.
Grok appeals to the caveman or woman in all of us. Cavemen, like us programmers, want powerful and flexible tools. Cavemen are great at tools after all; they invented the whole concept of them. But cavemen, and we, also want our tools to be simple and effective.
Cavemen want tools like clubs: a club is powerful, flexible (you can bash in anything, mash potatoes too) and also simple and effective. The ZTK is already powerful and flexible. Grok aims to make it simpler and more effective, for beginners and experienced developers alike. Grok: now even cavemen can use the ZTK.
The history of the ZTK is somewhat complicated. Since you will run into some of the older terminology involved when you read documentation, we will give a quick run-down here.
In the beginning (the late 1990s) there was just Zope. A version two of Zope was released quickly, but the fundamental nature of Zope 2 did not change. This version of Zope still is the basis of many projects, including the popular CMS Plone.
Then in the early 2000s the Zope community started a new project named "Zope 3". The original intent was for this project to eventually replace Zope 2, taking many lessons learned from Zope 2 into account.
Over time it became clear that Zope 2 was not so easily replaced. Instead, many Zope 3 libraries were adopted by Zope 2 developers and made an integral part of Zope 2 as well. Zope 2 and Zope 3 existed in parallel, sharing a common base. Then in 2006, Grok was built on top of Zope 3.
It became clear that Zope 3 was fulfilling two roles: as a foundational set of libraries for web frameworks, and as a web framework of its own. The naming of Zope 3 was confusing, as it suggested it being a successor to Zope 2 while it is not.
In 2009, we therefore reorganized the foundational set of libraries in Zope 3 as the Zope Toolkit. This is a base shared, at least in part, by Zope 2 and Grok. Finally in early 2010 the old Zope 3 was renamed BlueBream, which is also built on top of the ZTK base, just like Grok. Since BlueBream, Grok and Zope 2 all share a ZTK base, much code written for one project can, with some care, also be used in the other project.
To summarize the current state of affairs, there is a collection of libraries managed as the ZTK that forms the basis for Zope 2, BlueBream and Grok.
The ZTK allows you to combine different components in an explicit, flexible way. You can hook up a view to a model, an event handler to an event, and a new API to an existing object. The process of doing this is called configuration. In a technical sense, an important part of Grok can be understood as being an alternate configuration mechanism for the ZTK.
The standard ZTK approach is to use ZCML for hooking up objects together. ZCML is an XML-based configuration language. ZCML statements are stored in their own file, next to the code. While using ZCML has the benefit of being explicit and flexible, it can also make code harder to read as there are more files to consult in order to understand what is going on.
Grok does away with most ZCML. Instead it analyzes your Python code for the use of certain special base classes and directives, and then "groks" it. This grokking process results in the same configuration as it would have if you used the equivalent ZCML. We believe that having all configuration along with your Python code makes the code easier to follow and more fun to develop.
Grok has been designed so that if you organize your code in a certain way, you can even leave out most of the explicit directives in your Python code. This makes code written for Grok look clean and uniform. You still have all the power of explicit configuration available should you need it, however.
During the development of Grok we have taken a careful look at common patterns in ZTK-based code and configuration. Grok aims to make these patterns more easy to use and succinct.
Grok is a friendly caveman from the Stone Age.
He has a big club that he hunts mammoths with. He will also use this club to smash anything he doesn't like.
"ME GROK SMASH ZCML!"
The word grok comes from the novel Stranger in a Strange Land by Robert A. Heinlein, and is defined in his book as:
Grok means to understand so thoroughly that the observer becomes part of the observed - to merge, blend, intermarry, lose identity in group experience.