Preserve unicode when slugifying by default

[rtpg]

Thanks to @hwshim810 (for original implementation) and @lucemia
(for ping + pointing out the perf weirdness) from #572 for motivating this.

This change is motivated by the many many people using non-latin
languages who experience the (rather baffling) behavior of outright
stripping characters when generating slugs from stuff that doesn't
fit into ASCII.

We went through loads of pain as a programming community to get to
nicely supporting unicode everywhere, it's time to take advantage of
that fact and just let people have stuff appear in their native
language as much as possible.

[codecov]

Codecov Report

Merging #797 (cb9c75b) into master (83fe619) will increase coverage by 0.02%.
The diff coverage is 100.00%.

❗ Current head cb9c75b differs from pull request most recent head ddb5ce6. Consider uploading reports for the commit ddb5ce6 to get more accurate results

@@            Coverage Diff             @@
##           master     #797      +/-   ##
==========================================
+ Coverage   92.91%   92.94%   +0.02%     
==========================================
  Files           9        9              
  Lines         720      723       +3     
  Branches      140      141       +1     
==========================================
+ Hits          669      672       +3     
  Misses         34       34              
  Partials       17       17              

Impacted Files	Coverage Δ
taggit/models.py	98.29% <100.00%> (+0.04%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 83fe619...ddb5ce6. Read the comment docs.

[rtpg]

@lucemia would you be able to review thise changeset? given this is a pretty big change (in few lines of code of course) I do want to have at least one review from someone else

[lucemia]

@lucemia would you be able to review thise changeset? given this is a pretty big change (in few lines of code of course) I do want to have at least one review from someone else

Would love to, but I am not reviewer of this repo... is there a way to become a reviewer?

[rtpg]

Two options:

• you can just look at the code and make a comment. I want somebody to look over it but I am totally comfortable hitting the merge button if it looks good myself. You don't need to be a "real" reviewer.
• you can join the jazzband org (see here for more details) and do the review "for real"

[lucemia]

Two options:

• you can just look at the code and make a comment. I want somebody to look over it but I am totally comfortable hitting the merge button if it looks good myself. You don't need to be a "real" reviewer.
• you can join the jazzband org (see here for more details) and do the review "for real"

I joined Jazzband! thanks your advice

[rtpg]

#798 this PR seems to indicate another important change that might be necessary here: allow_unicode=True for SlugField. Will look at this issue to see if something should be done here

[rtpg]

Alright I made sure that SlugField would accept unicode, and improved the documentation. @lucemia if you could look over this one last time and we can merge it and then I'll prepare a release

[lucemia]

I checked the PR, it looks good to me!
Thanks your @rtpg nice work!

[jsonn]

Are you aware that the test case here is broken with Python 3.8?

[rtpg]

@jsonn Python 3.8 is within the test suite and is passing, do you have a stack trace showing failure that can be tracked down here?

[jsonn]

Traceback (most recent call last):
  File "/tmp/scratch/www/py-django-taggit/work/django-taggit-3.1.0/tests/test_models.py", line 27, in test_old_slugs
    self.assertEqual([tag.slug for tag in sample_obj.tags.all()], [""])
AssertionError: Lists differ: ['ai-ueo'] != ['']

First differing element 0:
'ai-ueo'
''

- ['ai-ueo']
+ ['']