If this is the way forward, I'd love for a better developer experience. I'm curr... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		Scene_Cast2 on Feb 3, 2025 \| parent \| context \| favorite \| on: Decorator JITs: Python as a DSL If this is the way forward, I'd love for a better developer experience. I'm currently wrangling with pytorch.compile (flex attention doesn't like bias terms - see issues 145869 and 144511 if curious). As much as I love speed and optimization, JIT (at least the pytorch flavor) currently has weird return types that break VS Code's intellisense, weird stack traces, limitations around printing, and random issues like the bias term, and limitations such as not supporting sparsity. Speaking of pytorch JIT workflows - what's a nice way of having a flag to turn off compilation?

singhrac on Feb 3, 2025 | [–]

You can decorate a function with @torch.compiler.disable() to disable a specific function (and anything further down the call stack).

dleeftink on Feb 3, 2025 | [–]

> random issues like the bias term

I'd like to know more about this!

Scene_Cast2 on Feb 3, 2025 | [–]

Basically this style of code - https://github.com/pytorch-labs/attention-gym/pull/84/files - has issues like this - https://github.com/pytorch/pytorch/pull/137452 https://github.com/pytorch/pytorch/issues/144511 https://github.com/pytorch/pytorch/issues/145869

For some higher level context, see https://pytorch.org/blog/flexattention/

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact