I have been "advocating" (i.e. complaining) for better REPLs basically the entire time I've been writing code - One of my favorite pasttimes is getting all flustered and trying to throw enough extensions and custom code to get an offline REPL.it style experience out of VS Code for whatever language I'm playing with at the moment, and lately its kinda been working (for Ruby, at least).

I am also very found of pointing out that Powershell still has one of the best ISE's/IDE's in the game for this kind of stuff. One window for editing, a terminal in which whatever you edit runs, and then access to every var/function/etc you just touched in that same terminal is a joyous experience that is shockingly hard to recreate for many languages. Hell half the time I'm just testing out random syntax or a portion of a function and being able to do that in the same session as the script I'm working is awesome.

With enough abuse VS code comes close (for Ruby at least, thank you Pry!) but it would be nice to get a similar experience with Python; I've used a few of the existing REPL options for python, but most of them require you to actually figure out pdb and even then it wasn't as tightly integrated as what I actually wanted.

I usually just have one terminal window open with the Python REPL, and another open with a text editor. Works fine for me, although admittedly it was slightly more convenient back when `reload` was a builtin. (That's maybe the one thing I miss from 2.x, if I had to pick one. Sure, it doesn't honestly really make sense as a builtin; but it could have been added to `site` or something.)

I feel like there’s a missing discussion as to why they aren’t going with Ipython

Would be interesting to their reasons, but I'd be surprised if they had chosen it. I freaking love ipython, but it has a bunch of dependencies and extends far beyond just being a repl for the language and introduces things like magic commands, execute as shell, fallback logic etc.

Given how tight python keeps it's standard library, it seems pretty much imposssible to imagine those kind of advance features being developed while providing the stability that python normally asks from it's standard library.

I just tested installing ipython and it came with 17 dependencies. Some of which are probably pretty heavy and/or way too in flux to make it into the standard library.

I'm sure it's for exactly the same reason that I'm often hesitant to install it - huge dependencies:

  $ pip show ipython
  [...]
  Requires: appnope, backcall, decorator, jedi, matplotlib-inline, pexpect, pickleshare, prompt-toolkit, pygments, stack-data, traitlets

My guess is that it's not really designed to be fully severed from Jupyter. It certainly shouldn't require Matplotlib to run a console REPL.

I am pretty sure matplotlib is not required. That is just a convenience library that improves the notebook experience if matplotlib is loaded. So you no longer need to run the magic function ‘%matplotlib inline’

While I haven’t figured out how, it is my understanding that there exists a workflow where you have ipython/jupyter installed once, and then have them use the Python interpreter and modules associated with each project.

But I’ve never figured it out and instead have a `requirements-dev.txt` with Jupyter and Ipython in every project because they are so good to have on hand when developing

Also, I'm genuinely thrilled to see cross-pollination from pypy back to cpython, so am actually really glad they did it this way - as cpython's JIT becomes production-ready, maybe more bits of python will become be self-hosting?

Also, jedi depends on parso. Parso does not yet support `match` (which is a big problem for me, as it means I need to switch AST library backing mutmut).

they did acknowledge some alternatives, but I agree more discussion would have been nice.

FTA - Option 3: Using other existing REPL implementations: The authors looked at several alternatives like IPython, bpython, ptpython, and xonsh. While all the above are impressive projects, in the end PyREPL was chosen for its combination of maturity, feature set, and lack of additional dependencies. Another key factor was the alignment with PyPy’s implementation.

IPython is huge: 15,5 Mo without any extras.

Python is 150mo. Hard to justify that 10% of the lines of code, source of bugs and maintenance only go to the shell.

I am curious about your suffixes.

I mean, I understand the post fine, but I have never seen the units (Mo, mo) before and am wondering where they came from.

I would guess mo is megabytes(megaoctets?) and Mo is gigabytes(is ipython really 15GB? yikes)

I think I remember seeing something about MB=Mo in french, the casing is probably safe to ignore in the gp as ipython is definitely not 15gb

Looks like it’s octets: https://en.m.wikipedia.org/wiki/Octet_(computing)

yeah sorry mb, french comming out

OT, but I love your newsletter!

Didn’t notice your username until you wrote “French”

I think ptpython would have been a much better choice. It is relatively small with few dependencies, much more feature complete, importantly it can run in windows terminal AFAIK which pyrepl can't at the moment. I suspect it has also seen much more testing because it is much more widely used.

This really seems like a missed opportunity, instead of another repl that will only be used by developers (they even stated that as primary motivation) who can't install anything else, they could have taken a repl that would actually be widely used to integrate into other programs... Instead I suspect pyrepl will eventually experience the same fate as the current repl, i.e. it will languish with no development and get replaced again eventually because it has become to painful to adjust to changes in the rest of the language and changes in terminals.

fewer dependencies than ipython perhaps, but still unacceptable for something that needs to be shipped as part of the language. also I feel that you are unduly pessimistic about its chances and that the PEP is right about this one - being written in python rather than C will get it a ton of contributions from the community if anything is found lacking. particularly since you won't necessarily need to know a ton about python internals to just contribute to the repl

The horrible startup time is probably part of it:

  $ time ipython3 -c 'pass'
  real    0m1.083s
  user    0m0.355s
  sys     0m0.093s

Is this worse than the standard python interpreter? Python startup time is a big reason I've moved a bunch of my personal utilities to Rust.

I think the issue with iPython is that it's really quire large and has a non-trivial number of third-party dependencies that would need to be dealt with.

Python already has a million libraries built in, adding more in would be a pain. Plus there is the politics of making something "core" when the maintainers aren't part of the core python team.

It's a good start, and please do more... Even small QOL improvements like tab completion for filenames are important and is what makes me to install ipython at this point.

Now that I've actually gone and read the PEP instead of just relying on my familiarity with the changes as described elsewhere (and experienced for myself):

... Wow, significant portions of that come across to me as AI-generated. I'm pretty sure this is the first time I've gotten that impression from a PEP.

This is great. Using the default REPL was always painful after getting used to ptpython. Looking forward to trying it!

I want a Lisp-like REPL for my Python programs that is available to the user who runs my compiled Python program (so, I want compilation as well). The user will be able to interact with my program (instead of just running the main function), change function definitions, etc. and mold the program to their specific use case while it's running.

>the user who runs my compiled Python program (so, I want compilation as well).

You'll be happy to know that Python .pyc files, which are created and cached by default, are the equivalent of Java's .class files or C#'s bytecode (which gets embedded inside an .exe wrapper but is still fundamentally not native code).

>The user will be able to... change function definitions, etc.

Of course, this requires recompilation in some form (in Lisp, too - `eval` is not that magic).

That said, if `import`ing your code at the REPL and then calling functions, setting attributes etc. (which has been possible in Python forever) isn't good enough, I really don't understand your use case.

Go with Smalltalk instead. Not only do you get REPL-level interactivity[1], user actions are also automatically persisted, and it works with a GUI by default! With Glamorous Toolkit (a Pharo Smalltalk-based IDE) you even get good interop with Python out of the box. Really, if you have users who would benefit from being able to interact with your code directly, Smalltalk is THE way to go :) Ofc, the problem is finding such users in the first place...

[1] As Smalltalk is most often used with GUIs, you don't get a "REPL" by default - instead, anywhere you can enter text, you can right-click and execute that text as code. You can code a real REPL yourself if you want - 10 lines in the "on new line character" method in a generic text field and you're done.

Some things are a bit awkward, but what specifically can't you really do in a Python REPL? You can dynamically overwrite functions in an imported module, you can re-import modules with importlib, etc. I ask because my Lisp experience is limited.

Lisp REPL keeps the state of the program because it is a live image. With Python REPL you need to rerun the program to set the variables to their values.

You can create a file, example.py:

    import time
    value = "foo"
    def go():
      for i in range(10):
        print("...", i, value)
        time.sleep(3)

Then, in repl,

    import example
    import threading
    thread = threading.Thread(target=example.go)
    thread.start()

It will slowly print out messages and you can do "example.value = 'bar'" in the REPL and it will change.

That's a nice trick!

For reference, what I'd normally do if I wanted that is specify launching with the debugger, like

    python3 -mpdb example.py

and then step through. Also `ipdb` is nice if available.

it's not the same though—In Lisp, when you compile your program, a Lisp run-time is attached to it so you can either run the program normally (like any binary) or you could REPL into the program and see what the values of variables are (these values were set at compile time). This is helpful when you have a large dataset you don't want to load over and over.

Repl can be attached to a live python process (you don't need to restart anything)

https://stackoverflow.com/questions/1395913/how-to-drop-into...

You could do it even without cooperation from the python app using approach similar to pyrasite (though it is much easier with cooperation--just add a couple of lines).

Have you tried out the Python standard library debugger (pdb)?

Something like https://hylang.org/ ? I suppose it won't work with compiled code though.

I don’t know how this works with the compilation angle, but this is often a life saver:

  try:
    # problem
  except:
    import code
    code.interact(local=locals())

You can add globals() in addition to locals() if desired.

This drops you into the interactive shell and you can go wild. Even works inside interactive code like grabbing a live HTTP request or GUI event to see what’s going on.

I replied here earlier with my own pet project to essentially say that you don't need a whole lot of complexity to solve for QOL improvements in the console but was promptly downvoted, I thought that was fair and so deleted my comment.

However, now I do feel the need to say this - you really do not need a whole lot to enhance your productivity on the python REPL by a large factor, if you take advantage of some simple built-in facilities:

A. Understand the use of the PYTHONSTARTUP environment variable. This alone is a big advantage. It'll allow you to you automatically import often needed modules and declare helpers that you always seem to need.

B. Once you've gotten used to that, Wrap the built-in module code (or I presume pyrepl) to add the little things that you need and point PYTHONSTARTUP to it

C. Enjoy

At the risk of being downvoted again (not that it matters, so won't delete this time), here again, shameless plug - https://github.com/lonetwin/pythonrc

They really don't. The Python userbase has a very high percentage of beginners at any time, and trying to move them into Jupyter or an IDE before they understand fundamentals, usually just makes it much harder to help them with programming problems (because they don't know what's language functionality and what comes from their tools; they don't know anything about the virtual environment the tool is managing; they aren't prepared for an IDE to identify problems that are different from what the runtime chokes on; etc.)

The basic REPL causes serious problems for beginners, and the differences in this REPL are (intentionally or not) largely geared towards fixing those problems. Most notably, beginners can't reliably paste in code examples from tutorials. Either there's a use of `input` which eats the next line of code as interactive input, or a blank line inside a block which the REPL interprets as end-of-code (causing the rest of the block to report `IndentationError`s despite being correctly indented for the intent of the code). They also commonly get tripped up by `help` and `exit` being Python callables rather than REPL commands, and get put off by the lack of built-in screen clearing. (Historically - as it turns out from forum discussion - the Python devs have expected that people actually quit the interpreter with ctrl-D/ctrl-Z, and clear the screen with the terminal emulator's functionality for doing so.)

If Python is going to include a REPL (which I think it should), it might as well include useful features that people expect from a modern REPL. (That being said, I personally always install ipython...)

There's a tremendous power with defaults and with "batteries included".

The same people removed distutils, which is why at my company we had to update several internal C-extensions.

In these decisions the only thing that matters is if Microsoft, Facebook, Bloomberg or one of their employees is pleased.

>The same people removed distutils

Yes; it's extremely cruft-filled and nobody wanted to / had the skills to keep the standard library version maintained. There's a note in the Setuptools documentation somewhere about how distutils was deemed fundamentally unfixable, and it doesn't come across like the hacks Setuptools was applying on top of distutils were particularly well understood by anyone.

>In these decisions the only thing that matters is if Microsoft, Facebook, Bloomberg or one of their employees is pleased.

I don't think there's good evidence for this, and the current case is certainly not convincing. Why would they want distutils removed - as opposed to that motivation come from the devs who would otherwise be responsible for its bugs?

It looks like a lot of care went in to disclosing this and providing replacements. Can I ask what you were using it for?

https://peps.python.org/pep-0632/

Not the one you are asking, but NumPy literally converted to meson because of the deprecation. There was tons of pain for so many extensions.

This pain generates job security for the bigcorp employees and grief for anyone else.

Super helpful, thanks! The NumPy page on the migration has some details on the differences:

    > ... [Here] are the numpy.distutils features that are not present in setuptools:
      * Nested setup.py files
      * Fortran build support
      * BLAS/LAPACK library support (OpenBLAS, MKL, ATLAS, Netlib LAPACK/BLAS, BLIS, 64-bit ILP interface, etc.)
      * Support for a few other scientific libraries, like FFTW and UMFPACK
      * Better MinGW support
      * Per-compiler build flag customization (e.g. -O3 and SSE2 flags are default)
      * a simple user build config system, see site.cfg.example
      * SIMD intrinsics support
      * Support for the NumPy-specific .src templating format for .c/.h files

https://numpy.org/doc/stable/reference/distutils_status_migr...

I never did understand this. It appears that they were maintaining their own fork of distutils anyway (and if you open up a latest-version wheel today you should still see it included); so why did the standard library removal cause a problem?

The deprecation of distutils caused a ton of havoc for e.g. FreeCAD

I won't argue with that, but I would like to understand why.

If I'm on the right path with these forum threads, it looks like there were issues with how Debian packages python?

https://forum.freecad.org/viewtopic.php?t=67985

https://github.com/FreeCAD/FreeCAD/pull/6753

https://ffy00.github.io/blog/02-python-debian-and-the-instal...

I don't know the details, all I know is because of that I can't currently install FreeCAD from Fedora repositories which is super annoying, and IMO not the fault of either Fedora or FreeCAD...

Maaaybe color highlight can annoy some purist, but multiline edit and cursor movement are the bare minimum. Either add these features or get rid of the battery included Repl in favour of third parties.

From my experience working with the default repl in e.g. a docker container is extremely frustrating comparing to ipython, and I don't want to be installing ipython and its multitude of dependencies everywhere

I've never actually used these industrial-strength "containers" before, but I had understood that they're supposed to be primarily for deployment, or at least some automated testing step in a big CI system. Why would the REPL come into play for these use cases?

> The new REPL released in Python 3.13 aims to provide modern features

No `vi` mode and not planned. Very modern.

https://github.com/python/cpython/issues/118840

Honestly, even as a neovim user, I don’t find vi mode to be very ergonomic for interactive prompts, and I prefer emacs-style keybindings in these cases. The only time I feel the need for vi mode is when I want to copy something, but in that case I already have that capability through tmux copy-mode. I would prefer if the team prioritizes python-specific functionality first and foremost.

I will be staying on ptpython then.

Yeah it is. Vi is ancient and not at all modern or use friendly. Look at that bug report! They're complaining they can't go up to the previous command by pressing ESC k. Instead they have to press.... up. Ye gads.