# Changing ByteStr REPR

A recent rebutal against Python 3 was recently written by the (in)famous Zed Shaw, with many responses to various arguments and counter arguments. One particular topic which caught my eye was the bytearray vs unicodearray debate. I'll try explicitely avoid the term str/string/bytes/unicode naming as it is (IMHO) confusing, but that's a debate for another time. If one pay attention to above debates, you might see that there are about two camps: bytearray and unicodearray are two different things, and we should never convert from one to the other. (that's rought the Pro-Python-3 camp) bytearray and unicodearray are similar enough in most cases that we should do the magic for users. I'm greatly exagerating here and the following is neither for one side or another, I have my personal preference of what I think is good, but that's irrelevant for now. Note that both sides argue that their preference is better for beginners. You can often find posts trying to explain the misconception string/str/bytes, like this one which keep insisting on the fact that str in python 3 is far different from bytes. The mistake in the REPR¶ I have one theory that the bytes/str issue is not in their behavior, but in their REPR. The REPR is in the end the main informatin communication channel between the object and the brain of the programmer, user. Also, Python "ducktyped", and you have to admit that bytes and str kinda look similar when printed, so assuming they should behave in similar way is not far fetched. I'm not saying that user will conciously assume bytes/str are the same. I'm saying that human brain inherently may do such association. From the top of your head, what does requests.get(url).content returns ? In [1]: import requests_cache import requests requests_cache.install_cache('cachedb.tmp') In [2]: requests.get('http://swapi.co/api/people/1').content Out[2]: b'{"name":"Luke Skywalker","height":"172","mass":"77","hair_color":"blond","skin_color":"fair","eye_color":"blue","birth_year":"19BBY","gender":"male","homeworld":"http://swapi.co/api/planets/1/","films":["http://swapi.co/api/films/6/","http://swapi.co/api/films/3/","http://swapi.co/api/films/2/","http://swapi.co/api/films/1/","http://swapi.co/api/films/7/"],"species":["http://swapi.co/api/species/1/"],"vehicles":["http://swapi.co/api/vehicles/14/","http://swapi.co/api/vehicles/30/"],"starships":["http://swapi.co/api/starships/12/","http://swapi.co/api/starships/22/"],"created":"2014-12-09T13:50:51.644000Z","edited":"2014-12-20T21:17:56.891000Z","url":"http://swapi.co/api/people/1/"}' ... bytes... I'm pretty sure you glanced ahead in this post and probaly thought it was "Text", even probably in this case Json. It might be invalid Json, I'm pretty sure you cannot tell. Why does it returns bytes ? Because it could fetch an image: In [3]: requests.get('https://avatars0.githubusercontent.com/u/335567').content[:200] Out[3]: b"\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x01\xcc\x00\x00\x01\xcc\x08\x06\x00\x00\x00X\xdb\x98\x86\x00\x00 \x00IDATx\xda\xac\xbdy\x93\x1b\xb9\xb2\xf6\xf7K\x00\xb5\x90\xbdH\xa3\x99\xb9s7\xbf\xf1:\x1c\x0e/\xdf\xff\xdb8\xec\xb0}\xd79g4Rw\xb3IV\x15\x80\xf4\x1f@\xedUl\xea\\w\x84\xa65-6Y\x85\x02ry\xf2\xc9'\xa5\xfe\x9f\xfeGE\x04#\x821\x061\x16c\x0c\xc6XD\x0c\x02\xa0\x8a\x8a\x801\xa4\x1f\x08\x880\xfdRUD\x04\xd5\xfe\xff#6z\x8c*\xaa\x82\x88\xe0C \x84@\xf7~\xa6yy\xc5=>Q>~\xe6\xe1\xf3g~\xfd\xa7\x7f\xc28\x07\xb6\x00\x84h-\x88A1(\xe0U\xd2\xfb\xb8t\r1(" And if you decode the first request ? In [4]: requests.get('http://swapi.co/api/people/2').content.decode() Out[4]: '{"name":"C-3PO","height":"167","mass":"75","hair_color":"n/a","skin_color":"gold","eye_color":"yellow","birth_year":"112BBY","gender":"n/a","homeworld":"http://swapi.co/api/planets/1/","films":["http://swapi.co/api/films/5/","http://swapi.co/api/films/4/","http://swapi.co/api/films/6/","http://swapi.co/api/films/3/","http://swapi.co/api/films/2/","http://swapi.co/api/films/1/"],"species":["http://swapi.co/api/species/2/"],"vehicles":[],"starships":[],"created":"2014-12-10T15:10:51.357000Z","edited":"2014-12-20T21:17:50.309000Z","url":"http://swapi.co/api/people/2/"}' Well that looks the same (except leading b...). Go explain a beginner that the 2 above are totally different things, while they already struggle with 0 base indexing, iterators, and the syntax of the language. Changing the repr¶ Lets revert the repr of bytesarray to better represent what they are. IPython allows to change object repr easily: In [5]: text_formatter = get_ipython().display_formatter.formatters['text/plain'] In [6]: def _print_bytestr(arg, p, cycle): p.text('') text_formatter.for_type(bytes, _print_bytestr) Out[6]: In [7]: requests.get('http://swapi.co/api/people/4').content Out[7]: Make a usefull repr¶ may not an usefull repr, so let's try to make a repr, that: Convey bytes are, in genral not text. Let us peak into the content to guess what it is Push the user to .decode() if necessary. Generally in Python objects have a repr which start with . As the _quoted representation of the object may be really long, we can ellide it. A common representation of bytes could be binary, but it's not really compact. Hex, compact but more difficult to read, and make peaking at the content hart when it could be ASCII. So let's go with ASCII reprentation where we escape non ASCII caracterd. In [8]: ellide = lambda s: s if (len(s) < 75) else s[0:50]+'...'+s[-16:] In [9]: def _print_bytestr(arg, p, cycle): p.text(''.format(hex(id(arg)))) text_formatter.for_type(bytes, _print_bytestr) Out[9]: In [10]: requests.get('http://swapi.co/api/people/12').content Out[10]: In [11]: requests.get('http://swapi.co/api/people/12').content.decode() Out[11]: '{"name":"Wilhuff Tarkin","height":"180","mass":"unknown","hair_color":"auburn, grey","skin_color":"fair","eye_color":"blue","birth_year":"64BBY","gender":"male","homeworld":"http://swapi.co/api/planets/21/","films":["http://swapi.co/api/films/1/","http://swapi.co/api/films/6/"],"species":["http://swapi.co/api/species/1/"],"vehicles":[],"starships":[],"created":"2014-12-10T16:26:56.138000Z","edited":"2014-12-20T21:17:50.330000Z","url":"http://swapi.co/api/people/12/"}' Advantage: It is not gobbledygook anymore when getting binary resources ! In [12]: requests.get('https://avatars0.githubusercontent.com/u/335567').content Out[12]:

# Remapping notebook shortcuts

As Jupyter notebook run in a browser for technical and practical reasons we only have a limited number of shortcuts available and choices need to be made. Often this choices may conflict with browser shortcut, and you might need to remap it. Today I was inform by Stefan van Der Walt that Cmd-Shift-P conflict for Firefox. It is mapped both to open the Command palette for the notebook and open a new Private Browsing window. Using Private Browsing windows is extremely useful. When developing a website you might want to look at it without being logged in, and with an empty cache. So let see how we can remap the Jupyter notebook shortcut. TL; DR; Use the following in your ~/.jupyter/custom/custom.js : require(['base/js/namespace'], function(Jupyter){ // we might want to but that in a callback on wait for // en even telling us the ntebook is ready. console.log('== remaping command palette shortcut ==') // note that meta is the command key on mac. var source_sht = 'meta-shift-p' var target_sht = 'meta-/' var cmd_shortcuts = Jupyter.keyboard_manager.command_shortcuts; var action_name = cmd_shortcuts.get_shortcut(source_sht) cmd_shortcuts.add_shortcut(target_sht, action_name) cmd_shortcuts.remove_shortcut(source_sht) console.log('== ', action_name, 'remaped from', source_sht, 'to', target_sht ) }) details We need to use require and register a callback once the notebook is loaded: require(['base/js/namespace'], function(Jupyter){ ... }) Here we grab the main namespace and name it Jupyter. Then get the object that hold the various shortcuts: var cmd_shortcuts = Jupyter.keyboard_manager.command_shortcuts. Shortcuts are define by sequence on keys with modifiers. Modifiers are dash-separated (need to be pressed at the same time). Sequence are comma separated. Example quiting in vim would be esc,;,w,q, in emacs ctrl-x,ctrl-c. Here we want to unbind meta-shift-p (p is lowercase despite shift being pressed) and bind meta-/ (The shortcut Stefan wants). Note that meta- is the command key on mac. We need to get the current command bound to this shortcut (cmd_shortcuts.get_shortcut(source_sht)). You could hardcode the name of the command but it may change a bit depending on notebook version (this is not yet public API). Here it is jupyter-notebook:show-command-palette. You now bind it to your new shortcut: cmd_shortcuts.add_shortcut('meta-/', action_name) And finally unbind the original one cmd_shortcuts.remove_shortcut('meta-shift-p') UI reflect your changes ! If you open the command palette, you should see that the Show command palette command now display Command-/ as its shortcut ! Future We are working on an interface to edit shortcuts directly from within the UI and not to have to write a single line of code ! Questions, feedback and fixes welcomed

# One less Pull Request Followup

My earlier blog post could not have been more timely, here is what I received this morning: Hacktoberfest is back! Ready to hack away? It’s that time of year again! Hacktoberfest 2016 is right around the corner and we’re back with new, featured projects and the chance to win the limited-edition Hacktoberfest T-shirt you all love. Read the blog post to see what’s changed this year and share on your favorite social media networks with #Hacktoberfest. Community Feedback. I quickly got some feedback in particular from Aaron Meurer. First there is no comment box on this blog. I tried it's painful to maintain, need moderation (...etc) so you can ping me on twitter, or open a GitHub. I think it's a high enough filter, if you have something to say to what I write here, you (likely) already have a GitHub account. Also, I ran a poll on twitter, with 41 responses, 51% (I assume 21 users) prefer few PRs. 49% (20 users) don't really count. So there is definitively a non-negligible population that will still be ok to contribute. This also explains Aaron tweet: if the number of pull requests discouraged pull requests we wouldn't have so many pull requests I would argue that if 50% of your users are not discouraged, you still discourage the other half. Which might be fine according to your own metrics. How to close ? Aaron expressed his concern that closing a PR without a comment might send the unintended message that: The maintainer does not want your code (assuming the maintainer closes) The Author does not want the project to get his code anymore (assuming the author closes) It might not have been clear enough in earlier post but when closing please, explain why you are closing, and what you expect. Here is one example of the IPython repository where the maintainers have close the PR: @takluyver and @ellisonbg have decided that we are going to close this PR and open an issue. We are still interested in this work going in, but the tests need to be written first. Feel free to re-open when the tests are ready. If you are closing your own work, please make it clear whether: You plan to work later on that If what you did can be reuse by future developers. When not to close Aaron pointed again that as a maintainer he prefers to keep PRs open. Now that GitHub allows you to give maintainers ability to push on your branch, You can give maintainers the ability to push on the PR. In case you disagree whether the PR should be close or not, exchange with the maintainer. The commits of a close PR can still be accessed and maybe a best course of action is for the maintainer to for your branch and re-issue the PR. They will have more control over it. Regardless discuss with the maintainer, convey your intent. Look deeper into maintainers habits. I purposely avoided the subject, but Andreas Mueller pointed out that you can actually look at recently merged PRs, and commit history. Only if all PR are old is Andreas discouraged. This is a valid strategy, but it requires time from the person that want to submit the PR. And it's not always easy to do. I tend to go this extra step when I really think the PR is worth it. Things are subjective Again, all these are personal thoughts and preferences. I prefer to have few PR, like I would prefer to have zero-inbox. I would be curious to see analysis of general type of contribution vs number of opened PRs. Are novice users less likely to contribute depending on the number of opened PRs? Are the structures of networks across project different ? Happy HacktoberFest.