Python 3 Metaclass Rambling

Posted in Uncategorized on January 18, 2018 by mwchase

It’s probably a good thing overall that the Python world shows such restraint, but I wonder if there are some very interesting possibilities we’re missing out on.

Heh. Heh-hahaha. Hahahahaha. A-HAHAHAHAHAHA—ahem.

Let’s port Haskell-ish type semantics into Python 3.

Some brief experimentation in the repl establishes that __getattribute__ is not a silver bullet, insofar as it doesn’t intercept calls such as to __getitem__ and __call__. Those appear to be resolved separately, using a parallel implementation of the descriptor protocol. Since Haskell’s… eccentrically expansive array of operators is unavailable anyhow, let’s focus on method call syntax.

So, what am I talking about with the idea of Haskell-ish semantics? Well, first off, let’s ditch inheritance. It’ll make the linters cry if we alter the semantics of the parens after a class statement, but I’m used to doing that to linters. Now, we’ll ditch inheritance, but not the idea of subtyping. It’ll just look different. My motivation here is to try to avoid creating weak typing through erasure; what I mean by this is that, for example, every instance of Nothing should know which type it belongs to.

One obvious detour to take is to question how immutability should enter into this situation. My inclination is to start with immutable values for the sake of simplicity, and just make it clear that this isn’t going to make nearby Python code Haskell-Approved by dint of proximity; this is basically a basic research project being done by one guy in his spare time.

To start with, algebraic data types. Python typing extensions have to fake forward typing with resolution tricks. I’d prefer something a little more explicit: ADTs can receive string-type arguments as placeholders, but they have to be explicitly bound after the referent is defined; that way, we know exactly what’s getting bound. Let’s sketch out a basic usage:

class Tree(ADT, a=..., Nil=(), Node=('a', 'Tree[a]', 'Tree[a]')):
    """Typical intro Haskell data structure"""

Tree['Tree'] = Tree

assert issubclass(Tree[int], Tree)
assert Tree[...] is Tree
assert isinstance(Tree[int].Nil(), Tree[int])

This isn’t getting into a lot of the more advanced stuff Haskell does, but let’s take stock.

The ADT convenience class has the metaclass we need to make everything else work. This metaclass takes the additional keyword arguments; one-letter lowercase names, with ellipsis value, for type variables, and uppercase names containing an argument tuple. In a more concrete type, some of these would actually have been defined at definition time, and could have been used directly. In any case, the string arguments can contain type variables, and forward definitions. Because they can contain stuff like Tree[a], they’ll actually need to be parsed into a little mini-AST that gets evaluated as bindings are done. The binding is accomplished with __setitem__, which associates the value of Tree with 'Tree' in Tree. Filling in type variables, on the other hand, is done with __getitem__. An ellipsis functions as a placeholder value in a multi-part index, as in NumPy; to support multi-part indices, non-tuple arguments must be converted to length-1 tuples. The constructors are available off the type, not sure if they can be referred to from the base type. They are callable with as many arguments as the length of the tuple. They MAY OR MAY NOT end up being classes themselves, and instances of the same constructor of the same type MAY OR MAY NOT be identical, or compare equal.

Another Haskell aside: do I want thunks? I’m always kind of a fan of thunks, so a better question is, do I particularly want thunks for this? I think I’d like to hold off, see if I can get Haskell-style function syntax crowbarred in.

I’ve decided I’m done for now, so I’ll inflict more of this on the world later.


Post Mortem Triple Feature: Rebirth

Posted in Uncategorized on January 3, 2018 by mwchase

One common thing that happens in human endeavors is a sequence of decisions, each seemingly sensible at the time, that end up leading to an overall bad outcome. That’s not what happened here. I started with a bad idea, and started piling patches onto it.

Here’s the current state of things:

  • The coding standard I developed, クソコード, proved to be approximately as bad as I was assuming it would be. The basic principal underlying it is, a software system consumes black-box APIs with internal code, and provides well-defined APIs based on that code. Now, there are all kinds of ways that could be expressed, but I chose to say “Hey, Python 3 allows programmers to access all Unicode letters when writing identifiers, and Japanese kana express the idea of ‘native concepts’ and ‘various things including loanwords’. What if we used them respectively for writing the identifiers from internal code and external libraries, and used latin letters for naming and invoking APIs, BUT we used fullwidth characters because Python 3 normalizes them to ASCII, and that looks funnier?”
    • The first problem with this approach is that fullwidth characters are not any integer multiple of the width of halfwidth characters in any of the GUI-based text editors I tried. So I decided to revisit Vim. More on that later.
    • A relatively minor problem is that this had me switching between three keyboard layouts in rapid succession.
    • There were several big problems with Python’s import system:
      • Some forms of the import statement are not subject to canonicalization, meaning that the import always fails.
      • Python 3 cannot import modules with accented characters in their name, on HFS+ volumes, unless you apply some Perfectly Normal™ monkey patches to the guts of the import system. Dakuten and handakuten count as “accents” for this purpose.
    • Final verdict on this experiment in self-torture: like Apps Hungarian Notation, it has some interesting ideas, and also like Apps Hungarian Notation, maybe you’d be better off working on the expressivity of the type system, rather than encoding type information in variable names.
  • I decided to do all of my Learning Experience stuff in a context where it’s fine if nobody else understands my code, like a coding challenge. I went for the Matasano crypto challenges, which I’d started once before, and then given up on, it turns out, because I wasn’t designing the overall system properly. I’ve technically solved Set 1 Challenge 6, but my current architecture/dataset doesn’t play as nice as I’d like with their suggested algorithm, so I’d like to do some theorycrafting on the initial winnowing step, try to get my solution faster and better-founded. Also, I hope there isn’t too much more Vanilla Ice.
    • For this attempt, I used the Click library, and I’d like to congratulate its maintainers for the elegance of their system, an elegance that showed through all of the stupid things I was attempting to do at the time.
  • Lastly, NeoVim: interesting software, I’d still like to learn it, but. The one-two punch of constantly switching keyboard layouts and trying to puzzle out logic errors was too much cognitive load. I could not learn Vim motions/text objects on top of that. I’ll need to find a less demanding context for learning this stuff.

Plans going forward:

  • Express the underlying concepts of クソコード in terms of the typing module, and figure out how to add Jedi support for stuff like NewTypes. (There’s hacked-in Union support, so it’s not unprecedented.)
  • Rewrite my current cryptopals code using pure ASCII.
  • Find something less challenging to write in NeoVim, like slideshows. Work my way up.

Adventures In Text Editing: Where Am I Now?

Posted in Uncategorized on December 26, 2017 by mwchase

dein is working great, and I’ve put together a good assortment of plugins so far, though I probably will install some more by the time I’m done. Current status:

  • tpope/vim-characterize
    • This adds extra functionality to the “ga” sequence.
  • sjl/gundo.vim
    • The documentation focuses on the :GundoToggle command, which I should figure out what I want to map it to before I need it, ideally. There are also a bunch of configuration variables I haven’t touched. (UPDATE: gundo doesn’t work with NeoVim, so I switched to simnalamburt/vim-mundo.)
  • Shougo/deoplete.nvim
    • I don’t see that this adds any commands. It does add the completion menu, of course. I haven’t used it in earnest in code, so I haven’t gotten a good feel for it, beyond that supertab didn’t seem to work well with it.
  • zchee/deoplete-jedi
    • Not much to say. There are options, I might mess with the python_path one.
  • rust-lang/rust.vim
    • I haven’t used this much yet. :RustFmt looks handy.
  • vim-syntastic/syntastic
    • I just made the recommended config changes.
  • vim-airline/vim-airline
    • I added some custom config to list the open buffers across the top of the screen.
  • mauromorales/vim-remark
    • Two documented commands: :RemarkBuild and :RemarkPreview. I haven’t bound either, or used remark much yet, so I’ll hold off for now.
  • hashivim/vim-vagrant
    • This adds a :Vagrant command with subcommands.
  • pearofducks/ansible-vim
    • There’s configuration. Haven’t touched it.
  • ctrlpvim/ctrlp.vim
    • I ended up going with this over NERDTree because it’s a lot easier thus far for me to deal with plugins that mess with vertical space rather than horizontal. There are three main commands documented, and I think I need more experience with the commands, and with binding, to decide what to do there.
  • tomtom/tcomment_vim
    • I need to understand vim better to know how this works.
  • michaeljsmith/vim-indent-object
    • “ai”, “ii”, “aI”. This is more stuff that I need to know vim better for.
  • tpope/vim-surround
    • The commands are cs<start sep><end sep>, ds<sep to remove>, ys<object to surround><sep to add>
  • mileszs/ack.vim
    • :Ack [options] {pattern} [{directories}], and I should :help Ack when I need it.
  • tpope/vim-eunuch
    • Installed because I was doing stuff like misnaming files. Copied command description from GitHub.
    • :Delete: Delete a buffer and the file on disk simultaneously.
    • :Unlink: Like :Delete, but keeps the now empty buffer.
    • :Move: Rename a buffer and the file on disk simultaneously.
    • :Rename: Like :Move, but relative to the current file’s containing directory.
    • :Chmod: Change the permissions of the current file.
    • :Mkdir: Create a directory, defaulting to the parent of the current file.
    • :Find: Run find and load the results into the quickfix list.
    • :Locate: Run locate and load the results into the quickfix list.
    • :Wall: Write every open window. Handy for kicking off tools like guard.
    • :SudoWrite: Write a privileged file with sudo.
    • :SudoEdit: Edit a privileged file with sudo.
  • sebastianmarkow/deoplete-rust
    • Added just now. It adds “gd” for “go to definition” and “K” for “show documentation”
  • mhinz/vim-signify
    • I added “autocmd BufRead,BufNewFile * setlocal signcolumn=yes” to my vimrc, now the sign column always shows up, so I’m fine using vim-signify.
  • lervag/vimtex
    • Impulse-added this just now because I don’t NOT like LaTeX. I’ll crack it open more later.

Hm. vim-surround plugs tpope/vim-repeat. I’ll need more experience to determine whether I want that.

Anyway, where this is now:

  • “ga” shows character information.
  • “gd” and “K” do Rust-related stuff now.
  • :MundoToggle is in relatively urgent need of a binding. The default binding from the doc doesn’t seem like a good idea for me.
  • A bunch of these plugins add commands that operate on “objects”, which I need to learn more about.

That’s probably all of the most urgent stuff.

Adventures In Text Editing: What’s Next

Posted in Uncategorized on December 22, 2017 by mwchase

Okay, so I’ve got NeoVim, tmux, and a bunch of ideas about what I want to install next.

First up, dein.vim. I need some kind of package manager, and dein sounds cool? I’m mostly just picking stuff that other people mention and going with it. Once I have dein working, I’ll be able to install a bunch of extensions, incrementally.

I’ll want support for the following file types:

  • Python (duh)
  • INI (tox is so great) (note from the future: here we can see that I didn’t bother to fire up NeoVim while I was listing these off)
  • Ruby (This isn’t super-important, but it’d be handy to be able to edit Vagrantfiles nicely)
  • YAML (Ansible is p. powerful)
  • Rust (the default config for the Rust stuff I have in Sublime Text tends to bring it to its knees, so I’m interested in trying out alternatives)
  • Markdown (the best alternative yet to PowerPoint, IMO)
  • HTML (I need it to put my Markdown in)
  • reStructuredText (maybe using it in a different editor will allow me to overcome my reflexive dislike of it? maybe?)

And I’ve got a bunch of extensions I want to try out, starting with Eevee’s vim bundle:

  • YouCompleteMe (general completion) (or maybe I’ll try deoplete)
  • ack (I’ll want to see how much I’m hurting for advanced grep, first.)
  • characterize (detailed character metadata)
  • ctrlp (scandal: I don’t really use this much in Sublime Text, even, so I’ll probably hold off)
  • gitgutter (actually, I’ll probably use signify, since I try to avoid git as much as possible) (“Max, those are all GitHub links.” On the internet, GitHub doesn’t know you’re using hg-git.)
  • gundo (vim’s undo system is apparently… quite powerful)
  • jedi (Python completion, integrates with YouCompleteMe)
  • markdown (probably just want to grab the vimrc modifications from the readme)
  • python-mode
  • rust
  • supertab
  • syntastic
  • airline (if I want this, I’ll probably try powerline, on the assumption that I’ll get nice behavior from tmux as well)
  • commentary (or tcomment)
  • indent-object
  • surround
  • unimpaired
  • mw-utils

I don’t know what some of those are properly for, so I’ll hold off on them to start with. I also found a few others that look promising:

These should do for a start. There are a few others that I might look into, but this is a good checklist to start with.

Adventures In Text Editing

Posted in Uncategorized on December 17, 2017 by mwchase

I recently came up with a deliberately awful code style convention as a joke, then decided to inflict it on myself. Until I reveal it formally, I’m going to refer to it as *****. How bad is it? Files written using ***** will fail to render properly in Sublime Text, Atom, Brackets, TextWrangler, LightTable, the ancient version of SubEthaEdit that I have kicking around, Textadept, Visual Studio Code, and sublemon. They come out fine when I cat them in iTerm, so it looks like I need a text editor with a terminal interface. When I need that, I reach for vim, but I would describe the state of my current vim installation as “not obviously as bad as it used to be”. Fine for quick editing tasks, but I’m not sure I didn’t somehow render it brittle over the years.

My plan is to use ***** for a simple hobby project, and see if it provides any insights. Because *****’s design does have somewhat legitimate motivations, it’s just that the execution is intentionally weird.

So, I’d like a new vim-esque program, to try out a clean slate kind of thing. I installed NeoVim, and it can render *****-compliant files fine, so I’m willing to give it a shot.

Now, I’m basically ignorant of what NeoVim (and to a large degree, vim and even vi) has to offer, so I’m going to put together a wishlist of features that may require plugins, but could well be built in.

  • Python 3 syntax support. I’m not aware of any other non-esoteric language that it’s possible to write code in ***** style. (EDIT: It’s actually possible to adhere to most of ***** in Java. I sort of knew this, completely forgot, and don’t care at all.) Now, I’ll have to take a look at exactly what any packages provide, because I now realize (after a coworker brought in a dependency that just markets itself by saying “[task] sucks. [library] makes it not suck”, which has no semantic content that I care about) that concepts like “support” can be pretty vague.
  • Semantic highlighting. I tried out a semantic highlighter in Sublime Text, and my only complaint so far is that it triggers in limited and hard-to-reason-about circumstances for Rust code. But in Python, I had no complaints, so I’m generally in favor of semantic highlighting now.
  • Some kind of autocomplete feature. There are several knobs to tweak in deciding exactly what ***** means. One of them, I chose to make source code look funnier, at the expense of breaking various utilities, such as autocomplete and docstring/signature inspection, for stdlib and third-party code. Nevertheless, ***** makes autocomplete very important for internal code, because some of the identifiers that ***** mandates mess with my laptop’s input handling code.
  • Other stuff that Sublime Text provides. NERDTree sounds cool. Mostly, I just need to start messing around.

I was looking into package managers, and it’s this weird bit of pointer-chasing, that led me to dein.vim, so I guess I’ll try that unless someone objects.

I also need to go through a NeoVim tutorial sometime.

UPDATE: The tutorial segfaulted. It told me to scroll down, I scrolled down, the display got all graphically glitchy, and then it segfaulted. This had better not be an omen.

UPDATE: The cursor keys corrupt the state in various bizarre ways. I’m going to check whether iTerm is somehow responsible. … IT IS. This is going to be a fun bug report.

UPDATE: I’m leaning towards blaming iTerm completely. Not sure how, but I managed to make things worse by toggling some iTerm settings back and forth.

UPDATE: The tutor also crashes under tmux in, but not on its own. Sure.


Posted in Uncategorized on December 12, 2017 by mwchase

Screen Shot 2017-12-11 at 8.49.00 PM

“Wh-what are you doing with that socket wrench and those jumper cables? Honey? Oh my god NOOOOooooooo”


Posted in Uncategorized on October 29, 2017 by mwchase

there was a controversy at that time over which one of twins was the elder: the one born first or the one who, being born second, would have, as was then thought, been conceived first

People used to think the womb was a stack-based data structure.