ModuleNotFoundError during py.import

[country-bumpkin-software]

Im trying to run this bit of code, inside my virtual env:

use criterion::{
    black_box, criterion_group, criterion_main, Bencher, Criterion,
  };
  use pyo3::prelude::*;
  use pyo3::types::PyDict;
  
  criterion_main!(python_vs_rust);
  criterion_group!(python_vs_rust, bench_fn);
  
  fn bench_py(b: &mut Bencher, code: &str, input: &str) {
    Python::with_gil(|py| {
      let locals = PyDict::new(py);
  
      locals.set_item("json", py.import("json").unwrap()).unwrap();
      locals.set_item("py_rust", py.import("py_rust").unwrap()).unwrap();
      locals.set_item("INPUT", input).unwrap();
  
      b.iter(|| black_box(py.run(code, None, Some(&locals)).unwrap()));
    });
  }
  
  fn bench_fn(c: &mut Criterion) {
    let input = r#"{"name": "lily", "value": 42}"#;
  
    c.bench_function("pure python", |b| {
      bench_py(
        b,
        "
  value = json.loads(INPUT)
  s = value['value'] + len(value['name'])
        ",
        input,
      );
    });
  
    c.bench_function("rust extension library", |b| {
      bench_py(b, "s = py_rust.sum(INPUT)", input);
    });
  }

But im getting a ModuleNotFoundError:

Gnuplot not found, using plotters backend
Benchmarking pure python: Warming up for 3.0000 sthread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: PyErr { type: <class 'ModuleNotFoundError'>, value: ModuleNotFoundError("No module named 'py_rust'"), traceback: None }', benches/py-vs-rust.rs:16:51
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Interestingly if I do the following :

@192-168-1-10 py_rust_benchmark % python 
Python 3.9.13 (main, May 24 2022, 21:13:51) 
[Clang 13.1.6 (clang-1316.0.21.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import py_rust
>>> print(py_rust)
<module 'py_rust' from '/Users//.local/share/virtualenvs/py_rust-QUdKXKyu/lib/python3.9/site-packages/py_rust/__init__.py'>
>>> 

I can import the module.

Can someone explain what im doing wrong?

I created the virtualenv with: pipenv --three and then Pipenv shell to activate it

[davidhewitt]

You probably need to set PYTHONHOME or PYTHONPATH environment variables before starting your program. I see you're running on macOS, so I think this is covered by #1741. The summary is that it's likely possible for PyO3 to cover this automatically, just someone needs to put some time in to research a robust implementation.

[davidhewitt]

https://docs.python.org/3/using/cmdline.html#envvar-PYTHONHOME

[country-bumpkin-software]

Thanks for the doco David.

Do I set the python home to be the python install in the virtual env that I ran Maturin with ?
For example my rust project I activated the pipenv and I get the (py_rust_project) if I do sys.executable. Is this the path I set in pythonhome?

Also if I try to do that then I can no longer open my virtualenv

[country-bumpkin-software]

Is there any further info you can provide? I got my companies Python expert to take a look at this and we still couldn't get it working

[davidhewitt]

e.g. put virtualenv at /Users/david/.virtualenvs/pyo3/

PYTHONPATH=/Users/david/.virtualenvs/pyo3/lib/python3.10/site-packages cargo run then works for me.

So I guess for you, try PYTHONPATH=/Users//.local/share/virtualenvs/py_rust-QUdKXKyu/lib/python3.9/site-packages ?

[country-bumpkin-software]

Thanks so much that resolved that error, it worked for me!