Getting Started

Installation

To install, use pip:

pip install redpipe

or from source:

python setup.py install

Get the Source Code

RedPipe is actively developed on GitHub.

You can either clone the public repository:

git clone git://github.com/72squared/redpipe.git

Or, download the tarball:

curl -OL https://github.com/72squared/redpipe/tarball/master

Once you have a copy of the source, install it into your site-packages easily:

python setup.py install

Connect redis-py to RedPipe

To use redpipe, You need to bind your redis client instance to RedPipe. Use the standard redis-py client.

client = redis.Redis()
redpipe.connect_redis(client)

You only need to do this setup once during application bootstrapping.

This example just sets one connection up as a default, since that is the most common case. But you can connect multiple redis connections to RedPipe.

You can use StrictRedis if you want too. It doesn’t matter. Whatever you use normally in your application.

The goal is to reuse your application’s existing redis connection. RedPipe can be used to build your entire in your application. Or you can use RedPipe along side your existing code.

More on this later.

Using RedPipe

Using RedPipe is easy. We can pipeline multiple calls to redis and assign the results to variables.

with redpipe.pipeline() as pipe:
    foo = pipe.incr('foo')
    bar = pipe.incr('bar)
    pipe.execute()
print([foo, bar])

RedPipe allocates a pipeline object. Then we increment a few keys on the pipeline object. The code looks mostly like the code you might write with redis-py pipelines. The methods you call on the pipeline object are the same. But, notice that each incr call immediately gets a reference object back in return from each call. That part looks similar to how redis-py works without a pipeline.

The variables (in this case foo and bar) are empty until the pipeline executes. if you try to do any operations on them beforehand, it will raise an exception. Once we complete the execute() call we can consume the pipeline results. These variables, foo and bar, behave just like the underlying result once the pipeline executes. You can iterate of it, add it, multiply it, etc.

Reusable Functions

You can write a function that takes in a pipeline, and returns a result before the pipeline even executes.

Here’s a quick example of what I mean:

def get_foo(pipe=None):
     with redpipe.pipeline(pipe=pipe) as pipe:
        pipe.setnx('foo', 'bar')
        foo = pipe.get('foo')
        pipe.execute()
        return foo

Now I can invoke my function. I can call it in isolation:

print(get_foo())

This will pipeline the following commands to redis:

  • SETNX foo bar
  • GET foo

Or I can pipeline it with other things:

with redpipe.pipeline() as pipe:
    foo = get_foo(pipe)
    bar = pipe.get('bar')
    pipe.execute()

This example will pipeline these three commands together:

  • SETNX foo bar
  • GET foo
  • GET bar

In this example, the foo and bar variables are both redpipe.Future objects. They are empty until the pipe.execute() happens outside of the function. The pipe.execute() called inside the get_foo function in this case is a NestedPipeline. It passes it’s stack of commands to the parent pipeline. That’s because we passed a pipeline object into the get_foo function. The function passed that into redpipe.pipeline and it returned a NestedPipeline to wrap the one passed in.