Coding Diet (testing)

unittest.mock small gotcha - a humbling tale of failure

Allan Clark — Sat, 25 Feb 2017 12:36:24 GMT

Developing a small web application I recently had reason to upgrade from Python 3.4 to Python 3.6. The reason for the upgrade was regarding ordering of keyword arguments and not related to the bug in my test code that I then found. I should have been more careful writing my test code in the first place, so I am writing this down as some penance for not testing my tests robustly enough.

A Simple example program

So here I have a version of the problem reduced down to the minimum required to demonstrate the issue:

import unittest.mock as mock

class MyClass(object):
    def __init__(self):
        pass
    def my_method(self):
        pass

if __name__ == '__main__':
    with mock.patch('__main__.MyClass') as MockMyClass:
        MyClass().my_method()
        MockMyClass.my_method.assert_called_once()

Of course in reality the line MyClass().my_method() was some test code that indirectly caused the target method to be called.

Output in Python 3.4

$ python3.4 mock_example.py
$

No output, leading me to believe my assertions passed, so I was happy that my code and my tests were working. As it turned out, my code was fine but my test was faulty. Here's the output in two later versions of Python of the exact same program given above.

Output in Python 3.5

$ python3.5 mock_example.py
Traceback (most recent call last):
  File "mock_example.py", line 12, in <module>
    MockMyClass.my_method.assert_called_once()
  File "/usr/lib/python3.5/unittest/mock.py", line 583, in __getattr__
    raise AttributeError(name)
AttributeError: assert_called_once

Assertion error, test failing.

Output in Python 3.6

$ python3.6 mock_example.py
Traceback (most recent call last):
  File "mock_example.py", line 12, in <module>
    MockMyClass.my_method.assert_called_once()
  File "/usr/lib/python3.6/unittest/mock.py", line 795, in assert_called_once
    raise AssertionError(msg)
AssertionError: Expected 'my_method' to have been called once. Called 0 times.

Test also failing with a different error message. Anyone who is (pretty) familiar with the unittest.mock standard library module will know assert_called_once was introduced in version 3.6, which is my version 3.5 is failing with an attribute error.

My test was wrong

The problem was, my original test was not testing anything at all. The 3.4 version of the unittest.mock standard library module did not have a assert_called_once. The mock, just allows you to call any method on it, to see this you can try changing the line:

        MockMyClass.my_method.assert_called_once()

        MockMyClass.my_method.blahblah()

With python3.4, python3.5, and python3.6 this yields no error. So in the original program you can avoid the calling MyClass.my_method at all:

if __name__ == '__main__':
    with mock.patch('__main__.MyClass') as MockMyClass:
        # Missing call to `MyClass().my_method()`
        MockMyClass.my_method.assert_called_once() # In 3.4 this still passes.

This does not change the (original) results, python3.4 still raises no error, whereas python3.5 and python3.6 are raising the original errors.

So although my code turned out to be correct (at least in as much as the desired method was called), had it been faulty (or changed to be faulty) my test would not have complained.

The Actual Problem

My mock was wrong. I should instead have been patching the actual method within the class, like so:

if __name__ == '__main__':
    with mock.patch('__main__.MyClass.my_method') as mock_my_method:
        MyClass().my_method()
        mock_my_method.assert_called_once()

Now if we try this in all version 3.4, 3.5, and 3.6 of python we get:

$ python3.4 mock_example.py 
$ python3.5 mock_example.py 
Traceback (most recent call last):
  File "mock_example.py", line 12, in <module>
    mock_my_method.assert_called_once()
  File "/usr/lib/python3.5/unittest/mock.py", line 583, in __getattr__
    raise AttributeError(name)
AttributeError: assert_called_once
$ python3.6 mock_example.py 
$

So Python 3.4 and 3.6 pass as we expect. But Python3.5 gives an error stating that there is no assert_called_once method on the mock object, which is true since that method was not added until version 3.6. This is arguably what Python3.4 should have done.

It remains to check that the updated test fails in Python3.6, so we comment out the call to MyClass().my_method:

$ python3.6 mock_example.py 
Traceback (most recent call last):
  File "mock_example.py", line 12, in <module>
    mock_my_method.assert_called_once()
  File "/usr/lib/python3.6/unittest/mock.py", line 795, in assert_called_once
    raise AssertionError(msg)
AssertionError: Expected 'my_method' to have been called once. Called 0 times.

This is the test I should have performed with my original test. Had I done this I would have seen that the test passed in Python3.4 regardless of whether the method in question was actually called or not.

So now my test works in python3.6, fails in python3.5 because I'm using the method assert_called_once which was introduced in python3.6. Unfortunately it incorrectly passes in python3.4. So if I want my code to work properly for python versions earlier than 3.6, then I can essentially implement assert_called_once() with assert len(mock_my_method.mock_calls) == 1. If we do this then my test passes in all three version of python and fails in all three if we comment out the call MyClass().my_method().

Conclusions

I made an error in writing my original test, but my real sin was that I was a bit lazy in that I did not make sure that my tests would fail, when the code was incorrect. In this instance there was no problem with the code only the test, but that was luck. So for me, this served as a reminder to check that your tests can fail. It may be that mutation testing would have caught this error.

In what ways are dynamically typed languages more productive?

Allan Clark — Wed, 04 May 2016 19:41:57 GMT

Introduction

I do not aim to answer the question in the title more raise the question. The question kind of implies that dynamically typed languages are more productive in at least some ways. It does not imply that statically typed languages are less productive in general, or the opposite.

Before going any further, I'm talking about the distinction between static and dynamic typing which is not the same as strong vs weak typing. Static means the type checking is done at compilation before the program is run, whilst dynamic means types are checked whilst the program is running. Not the same as weak vs strong typing and also not the same as explicit vs implicit typing.

Background

My PhD involved the development of a novel static type system. When I begun my PhD I was firmly of the opinion that statically typed languages were better than dynamically typed languages. My basic contention was that all the guarantees you get from static types you get largely for free, so throwing those away is a stupefying act of self harm. I believe that prior to the turn of the century (if not a bit later), this the majority view in programming language research. It is still a position commonly held but I'm unsure whether it may still be a majority view or not.

New data, should force us to seek out new theories which explain all of the available data. In the past 20 years, one thing is obvious. Dynamically typed languages have seen significant growth and success. Some researchers choose to ignore this. Some explain the success as a fluke, that such languages have become popular despite being dynamically typed. This is a recurrence of an argument made by functional programmers/researchers when C++ and then Java became wildly popular.

The prevailing view amongst researchers was that functional languages were inherently better than Java, but Java got a lot of financial support which meant that a lot of support was added to, in particular, the standard library. This meant that programmers were particularly productive using Java, but that that increase in productivity was mis-attributed by the programmers to the language, when it should have been placed firmly in the excellent standard library support. Much of this support is work that open source programmers are not quite so keen on, because frankly, it's a bit boring and unrewarding. I personally find this argument at least lightly compelling.

However, in the first decade of this century, as I said, dynamically typed languages have had at least significant success. Python, PHP, and Ruby are the most obvious examples. None of these were backed financially by any large corporation, at least not prior to their success. I again suspect that much of the productivity gained with the use of such languages can be placed in the library support. But that does not explain where the library support has come from. If dynamically typed languages were so obviously counter-productive, then why did anyone waste their time writing library support code in-and-for them?

Some Wild Hypotheses Appear

I am now going to state some hypotheses to explain this. This does not mean I endorse any of these.

Short term vs Long Term

One possible answer. Dynamically typed languages increase short term productivity at the cost of long term productivity. I don't personally believe this but I do find it plausible. However, I do not know of any evidence for or against this position. I'm not even sure there is much of a logical argument for it.

The kinds of bugs that functional programming languages help prevent are the kind of bug that is hard to demonstrate. It is easy enough to show a bug in an imperative program that would not occur in a functional program because you do not have mutable state. However, such demonstration bugs tend to be a bit contrived, and it is hard to show that such bugs come up in real code frequently. On top of that, to show that functional languages are more productive one would have to show that by restricting mutable state you do not lose more productivity than you gain by avoiding such bugs. If you did manage to show this, you would have a reasonable argument that functional languages are bad for short-term productivity, due to the restrictions on mutable state changes, but compensate in greater long-term productivity.

So, a similar kind of argument could be made for statically typed languages. If you could show that statically typed languages prevent a certain class of bugs and that the long-term productivity gained from that is more than enough to compensate for any short-term loss in productivity brought on by restrictions due to the type-system.

So I will leave my verdict on this hypothesis as, I believe it to be false but it is plausible. Just to note, there is no great evidence that either statically typed languages have greater long-term productivity, or that dynamically typed languages have greater short-term productivity.

Testing Tools

A trend that seems to have tracked (ie. correlates with, either via causation in either direction or by coincidence) the trend in use of (and success of) dynamically typed languages is the trend towards more rigorous testing, or rather the rise in popularity of more rigorous testing. In particular test-driven development style methodologies have gained significant support.

I believe that having a comprehensive test suite, somewhat dilutes the benefits gained from a static type system. Here is a good challenge, try to find a bug that is caught by the type checker, that would not be caught by a test suite with 100% code coverage.

One possibility is an exhaustive pattern match, however, if your test suite is not catching this, it's not a great test suite. Still, exhaustive pattern match tests is something you get more or less for free with a static type checker, whilst a test suite has to work at it.

It is certainly possible to come up with a bug that is caught by static type checking and not by a test suite that has full coverage. Here is an example:

    if (a and b):
        x = 1
    else
        x = "1"
    if b:
        return string(x + 2)
    else:
        return x

This is a bug, because b might be true, whilst a is false. Which would mean that x is set to a string, but later treated as an integer, because b is true. A good test suite, will of course catch this bug. But it is still possible to achieve 100% code coverage (at the statement) level, and not catch this bug. Still, you have to try quite hard to arrange this.

Mutation testing, which tests your tests, rather than your implementation code, should catch this simple example (because it will mutate the condition (a and b) to be (a or b) which won't make any difference if your tests never caught the bug initially. This will mean that the mutant will pass all tests, and you should at that point realise your tests are not comprehensive enough.

Dynamically Typed Language Benefits

So you should have a comprehensive test suite for all code whether you are using a statically or dynamically typed language. We may then accept the theory that a comprehensive test suite somewhat dilutes any benefits of using a statically typed language. However, that theory does not give any reasaon why a static type system is detrimental to productivity.

This was always my main contention, that whatever might be the benefits of static typing, you are getting them for free, so why not? I honestly do not know what, if any, benefit there is from having a dynamic type system. I can think of some plausible candidates, but have no evidence that any of these are true:

Freeing the language from a type system, allows the language designers to include some productivity boosting features that are not available in a statically typed language. I find this suggestion a little weak, no one has ever been able to point me to such a language feature.
Knowing there is no safety net of the type system encourages developers to test (and document) more. I find this theory more compelling.
One can try simple modifications without the need to make many fiddly changes, such as, for example, adding an exception to a method signature, often in many places.

I suspect, that if there are significant benefits to using a dynamically typed language, then it is a combination of 2 and 3, or some other reason.

For the third, a rebuttal often mentions automatic refactoring tools. Which may well in theory be something of a good rebuttal, but in practice developers simply don't use such tools often. I'm not sure why not, I myself have never taken to them. So perhaps there is a productivity gain from using a dynamically typed language which would be all but negated if only the developers would use automatic refactoring tools, but they don't. So it shouldn't be a productivity win for dynamically typed languages, but in practice it is (this is all still conjecture).

The second one has some evidence from psychology. There is a lot of evidence to suggest that safety mechanisms often do not increase overall safety but simply allow more risky behaviour. A very famous example is a seat-belt study done in Germany. Wherein mandating the wearing of seat-belts caused drivers to drive faster. This means that you are more likely to have a crash, but less likely to be seriously injured in one. This has similarly been done with anti-lock braking systems, where the brakes being significantly better did not reduce accidents but rather increased risky driving so that the number of accidents remained largely constant.

I mentioned documentation because it's an important one. There are plenty of libraries for statically typed languages for which the only documentation for many of the functions/methods is the type signature. This is often seen as "good enough", or at least good enough to mean that API documentation is not at the top of the todo stack. A dynamically-typed language does not typically have signatures for methods/functions. As a result, they tend to have fewer undocumented libraries, simply because the developer of the library knows that their methods will otherwise not be used. If that is the case, what is the point in the library? So they tend to write something, and once you are writing something it isn't so hard to write something useful.

Summary

This is getting too long. So I'll stop there for now. The main points are:

Dynamically typed languages have had a lot of success this century, which remains largely unexplained
I think that with comprehensive testing, the gains from a static type system are diluted significantly
It might be that dynamically typed language encourage more/better testing (or have some other non-obvious advantage)
Otherwise, there is scant evidence for anything a dynamically typed language actually does better
I think an obvious win for statically typed languages would be to make the type checking phase optional.

I am not sure why we do not really see any examples of languages that have optional static type checking. In other words languages in which the user decided when type checking should be done.

As a final point. For dynamically typed languages, it is common to deploy some form of static analyser. Often these static analysers fall short of the kind of guarantees afforded by a static type system, but they have two significant advantages. Firstly, you can run the program without running the static analyser. In particular, you can run your test suite, which may well give you more information about the correctness of your code than a type checker would. Especially in the case that the type checker fails. It tells you about one specific problem in your code, but not how the rest of your code does or does not pass your tests. Secondly you can deploy different static analysers for different problems. For example a statically typed language has to decide whether or not to include exceptions in types. A dynamically typed language can easily offer both. I suppose a statically typed language could offer both as well.

Selenium and Javascript Events

Allan Clark — Wed, 16 Mar 2016 18:02:32 GMT

Selenium is a great way to test web applications and it has Python bindings. I explained in a previous post how to set this up with coverage analysis.

However, writing tests is non-trivial, in particular it is easy enough to write tests that suffer from race conditions. Suppose you write a test that includes a check for the existence of a particular DOM element. Here is a convenient method to make doing so a one-liner. It assumes that you are within a class that has the web driver as a member and that you're using 'pytest' but you can easily adapt this for your own needs.

def assertCssSelectorExists(self, css_selector):
    """ Asserts that there is an element that matches the given
    css selector."""
    # We do not actually need to do anything special here, if the
    # element does not exist we fill fail with a NoSuchElementException
    # however we wrap this up in a pytest.fail because the error message
    # is then a bit nicer to read.
    try:
        self.driver.find_element_by_css_selector(css_selector)
    except NoSuchElementException:
        pytest.fail("Element {0} not found!".format(css_selector))

The problem is that this test might fail if it is performed too early. If you are merely testing after loading a page, this should work, however you may be testing after some click by a user which invokes a Javascript method.

Suppose you have an application which loads a page, and then loads all comments made on that page (perhaps it is a blog engine). Now suppose you wish to allow re-loading the list of comments without re-loading the entire page. You might have an Ajax call.

As before I tend to write my Javascript in Coffeescript, so suppose I have a Coffeescript function which is called when the user clicks on a #refresh-comment-feed-button button:

refresh_comments = (page_id) ->
  posting = $.post '/grabcomments', page_id: page_id
  posting.done receive_comments
  posting.fail (data) ->
    ...

So this makes an Ajax call which will call the function receive_comments when the Ajax call returns (successfully). We write the receive_comments as:

receive_comments = (data) ->
  ... code to delete current comments and replace them with those returned

Typically data will be some JSON data, perhaps the comments associated with the page_id we gave as an argument to our Ajax call.

To test this you would navigate to the page in question and check that there are no comments, then open a new browser window and make two comments (or alternatively directly adding the comments to the database), followed by switching back to the first browser window and then performing the following steps:

    refresh_comment_feed_css = '#refresh-comment-feed-button'
    self.click_element_with_css(refresh_comment_feed_css)
    self.check_comments([first_comment, second_comment])

Where self.check_comments is a method that checks the particular comments exist on the current page. This could be done by using find_elements_by_css_selector and then looking at the text attributes of each returned element.

The problem is, that the final line is likely to be run before the results of the Ajax call invoked from the click on the #refresh-comment-feed-button are returned to the page.

A quick trick to get around this is to simply change the Javascript to somehow record when the Ajax results are returned and then use Selenium to wait until the relevant Javascript evaluates to true.

So we change our receive_comments method to be:

comments_successfully_updated = 0
receive_comments = (data) ->
  ... code to delete current comments and replace them with those returned
  comments_successfully_updated += 1

Note that we only increment the counter after we have updated the page.

Now, we can update our Selenium test to be:

    refresh_comment_feed_css = '#refresh-comment-feed-button'
    self.click_element_with_css(refresh_comment_feed_css)
    self.wait_for_comment_refresh_count(1)
    self.check_comments([first_comment, second_comment])

The 1 argument assumes that this will be the first time the comments are updated during your test. Of course as you run down your test you can increase this argument as required. The code for the wait_for_comment_refresh_count is given by:

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions
from selenium.webdriver.common.by import By
...

class MyTest(object):
    ... # assume that 'self.driver' is set appropriately.
    def wait_for_comment_refresh_count(self, count):
        def check_refresh_count(driver):
            script = 'return comments_successfully_updated;'
            feed_count = driver.execute_script(script)
            return feed_count == count
        WebDriverWait(self.driver, 5).until(check_refresh_count)

The key point is executing the Javascript to check the comments_successfully_updated variable with driver.execute_script. We then use a WebDriverWait to wait for a maximum of 5 seconds until the our condition is satisfied.

Conclusion

Updating a Javascript counter to record when Javascript events have occurred can allow your Selenium tests to synchronise, that is, wait for the correct time to check the results of a Javascript event.

This can solve problems of getting a StaleElementReferenceException or a NoSuchElementException because your Selenium test is running a check on an element too early before your page has been updated.

Test First and Mutation Testing

Allan Clark — Fri, 19 Feb 2016 10:14:43 GMT

Test First and Mutation Testing

I'm going to argue that mutation testing has a strong use in a test first development environment and I'll conclude by proposing a mechanism to link mutation testing to the source code control mechanism to further aid test first development.

Test First

Just to be clear, when I say 'test first' I mean development in which before writing a feature, or fixing a bug, you first write a test which should only pass once you have completed that feature. For the purposes of this post you needn't be doing that for every line of code you write. The idea here applies whether you are writing the odd feature by first writing a test for it, or whether you have a strict policy of writing no code until there is a test for it.

Mutation Testing

Mutation testing is the process of automatically changing some parts of your source code generally to check that your test suite is not indifferent to the change. For example, your source code may contain a conditional statement such as the following:

    if x > 0:
        do_something()

Now if we suppose that the current condition is correct, then changing it to a similar but different condition, for example x >= 0 or x > 1 then presumably this would turn correct code into incorrect code. If your tests are comprehensive then at least one of them should fail due to the now incorrect code.

Fail First

It's easy enough to unintentionally write a test that always passes, or perhaps passes too easily. One of the reasons for writing the test first is to make sure that it fails when the feature has not yet been implemented (or fixed). However, often such a test can fail for trivial reasons. For example you may write a unit test that fails simple because the method it tests is not yet defined. Similarly a web test may fail because the route is not yet defined. Unless you continue to run the test during development of your feature you won't necessarily know that your test is particularly effective at catching when your feature is broken.

Fail After

Whether you write the test before your new feature or after the feature is ready, mutation testing can assist with the problem of non-stringent tests. Mutation testing can assist in reassuring you that your new test is effective at catching errors, whether those errors are introduced when the feature is developed or through later changes. If you apply lots of mutations to your code and your new test never fails then there is a strong likelihood that you have an ineffective test that passes too easily.

Source Code control

A feature I would like to add to a mutation test package is to integrate with a source code control mechanism such as Git. The mutation tester must choose lines of the program to mutate. However, your new test is presumably aimed at testing the new code that you write. Hence we could use the source code control mechanism to mutate lines of code that are newer than the test or some specified commit. That way we would focus our mutation testing to testing the efficacy of the new test(s) with respect to the new or changed lines of code.

This does not preclude doing general mutation testing for features that, for example, depend upon a lot of existing code. Perhaps your new feature is simply a display of existing calculations.

Conclusion

In summary:

Mutation testing helps find tests that are ineffective.
This plays particularly well with a test first development process in which the test often fails the first time for trivial reasons, thus giving you false assurance that your test can fail.
Integrating source code control to target the mutations towards new code could improve this significantly, or at least make it a bit more convenient.

Update: Flask+Coverage

Allan Clark — Tue, 26 Jan 2016 14:59:44 GMT

Update: Flask+Coverage Analysis

In a previous post I demonstrated how to get coverage analysis working for a Flask web application in a relatively simple manner. In the section "At then end of your tests" I stated that you needed your tests to clean-up by telling the server to shutdown. The end of your test code would look something like this:

finally:
    driver.get(get_url('shutdown'))
    ...

This could have made things a little fiddly since your test code would have to make sure to access the shutdown route exactly once, regardless of how many tests were run.

However, I realised that we could remove the burden from the test code by simply doing this in manage.py file.

Updated `manage.py`

Previously, we had the following code within our manage.py script within the run_with_test_server method:

test_process = subprocess.Popen(test_command)
test_process.wait(timeout=60)
server_return_code = server.wait(timeout=60)

We now update this to be:

test_process = subprocess.Popen(test_command)
test_process.wait(timeout=60)
port = application.config['TEST_SERVER_PORT']
shutdown_url = 'http://localhost:{}/shutdown'.format(port)
response = urllib.request.urlopen(shutdown_url)
print(bytes.decode(response.read()))
server_return_code = server.wait(timeout=60)

Doing so means you can just write your tests without any need to worry about shutting down the server. The example repository has been appropriately updated.

Flask + Coverage Analysis

Allan Clark — Mon, 25 Jan 2016 15:20:50 GMT

Flask + Coverage Analysis

This post demonstrates a simple web application written in Flask with coverage analysis for the tests. The main idea should be pretty translatable into most Python web application frameworks.

Update

I've updated this scheme and described the update here.

tl;dr

If you're having difficulty getting coverage analysis to work with Flask then have a look at my example repository. The main take away is that you simply start the server in a process of its own using coverage to start it. However, in order for this to work you have to make sure you can shut the server process down from the test process. To do this we simply add a new "shutdown" route which is only available under testing. Your test code, whether written in Python or, say Javascript, can then make a request to this "shutdown" route once it completes its tests. This allows the server process to shutdown naturally and therefore allow 'coverage' to complete.

Introduction

It's a good idea to test the web applications you author. Most web application frameworks provide relatively solid means to do this. However, if you're doing browser automated functional tests against a live server I've found that getting coverage to work to be non-trivial. A quick search will reveal similar difficulties such as this stack overflow question, which ultimately points to the coverage documentation on sub-processes.

Part of the reason for this might be that the Flask-Testing extension provides live server testing class that starts your server in testing mode as part of the start-up of the test. It then also shuts the server process down, but in so doing does not allow coverage to complete.

A simpler method is to start the server process yourself under coverage. You then only need a means to shutdown the server programatically. I do this by adding a shutdown route.

Selenium vs CasperJS

Allan Clark — Fri, 08 Jan 2016 11:11:40 GMT

Suppose you have a web service using a Python web framework such as Flask. So you wish to do a full user test by automating the browser. Should you use the Python bindings to Selenium or CasperJS? In this post I wish to detail some of the advantages and drawbacks of both.

Python + Selenium Setup

This is fairly straightforward. You are simply writing some Python code that happens to call a library which binds to Selenium. In order for this to work you will need to install phantomJS but using npm this is pretty trivial.

Javascript + CasperJS Setup

I say Javascript, but at least when I'm using casperJS it tends to be from Coffeescript, but whichever is your fancy works well enough. Similarly to the above you will need to install casperJS through the npm but again this is pretty trivial.

Speed

For my sample project the casperJS tests are faster, by quite a bit. This certainly warrants some more investigation as to exactly why this is the case, but for now my sample project runs the casperJS tests in around 3 seconds whilst the Selenium ones take around 12 seconds to do the same amount of work. I'm not sure whether this is a constant factor or whether it will get worse with more tests.

Coding Diet (testing)

unittest.mock small gotcha - a humbling tale of failure

A Simple example program

Output in Python 3.4

Output in Python 3.5

Output in Python 3.6

My test was wrong

The Actual Problem

Conclusions

In what ways are dynamically typed languages more productive?

Introduction

Background

Some Wild Hypotheses Appear

Short term vs Long Term

Testing Tools

Dynamically Typed Language Benefits

Summary

Selenium and Javascript Events

Conclusion

Test First and Mutation Testing

Test First and Mutation Testing

Test First

Mutation Testing

Fail First

Fail After

Source Code control

Conclusion

Update: Flask+Coverage

Update: Flask+Coverage Analysis

Updated manage.py

Flask + Coverage Analysis

Flask + Coverage Analysis

Update

tl;dr

Introduction

Selenium vs CasperJS

Python + Selenium Setup

Javascript + CasperJS Setup

Speed

Updated `manage.py`