TechHub

1 post1 participant0 posts today

narkodeI am transforming a project from <a href="https://nerdculture.de/tags/pandas" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#pandas</a> to <a href="https://nerdculture.de/tags/polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#polars</a> and I already like it! I don't want to talk bad about <a href="https://nerdculture.de/tags/pandas" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#pandas</a>, it did a lot for the data science community. But, beside of being blazingly fast, the syntax of <a href="https://nerdculture.de/tags/polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#polars</a> is much more logic and consistent!

Towards Data ScienceStruggling with slow GIS processing? Tony Albanese's article demonstrates a lightning-fast method for generating transects using <a href="https://hachyderm.io/tags/Geopandas" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Geopandas</a> and <a href="https://hachyderm.io/tags/Polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Polars</a>, reducing processing time from days to seconds. <a href="https://towardsdatascience.com/harnessing-polars-and-geopandas-to-generate-millions-of-transects-in-seconds-d37b176a0b57/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://towardsdatascience.com/harnessing-polars-and-geopandas-to-generate-millions-of-transects-in-seconds-d37b176a0b57/</a>

PyCharm BlogPolars vs. pandas – Python のデータフレームライブラリを徹底比較 <a href="https://techhub.social/tags/Python" class="mention hashtag" rel="tag">#Python</a> <a href="https://techhub.social/tags/Pycharm" class="mention hashtag" rel="tag">#Pycharm</a> <a href="https://techhub.social/tags/Datascience" class="mention hashtag" rel="tag">#Datascience</a> <a href="https://techhub.social/tags/Pandas" class="mention hashtag" rel="tag">#Pandas</a> <a href="https://techhub.social/tags/Polars" class="mention hashtag" rel="tag">#Polars</a><a href="https://blog.jetbrains.com/pycharm/2025/02/polars-vs-pandas" target="_blank" rel="nofollow noopener noreferrer" translate="no">https://blog.jetbrains.com/pycharm/2025/02/polars-vs-pandas</a>

CuducosDo I know any <a href="https://tech.lgbt/tags/Polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Polars</a> contributor or maintainer? I would love to help with that bug/exploration, but I am not sure where to start!<a href="https://github.com/pola-rs/polars/issues/21851" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://github.com/pola-rs/polars/issues/21851</a> <a href="https://tech.lgbt/tags/Python" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Python</a> <a href="https://tech.lgbt/tags/Rust" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Rust</a>

EuroSciPyWorking on core array computing libraries that power <a href="https://fosstodon.org/tags/scientificPython" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#scientificPython</a>? <a href="https://fosstodon.org/tags/EuroSciPy2025" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#EuroSciPy2025</a> wants your proposals on optimized array operations, vectorization techniques, and numerical foundations. Submit your groundbreaking work!<a href="https://pretalx.com/euroscipy-2025/cfp" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://pretalx.com/euroscipy-2025/cfp</a><a href="https://fosstodon.org/tags/ScientificComputing" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ScientificComputing</a> <a href="https://fosstodon.org/tags/Python" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Python</a> <a href="https://fosstodon.org/tags/EuroSciPy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#EuroSciPy</a> <a href="https://fosstodon.org/tags/numpy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#numpy</a> <a href="https://fosstodon.org/tags/scipy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#scipy</a> <a href="https://fosstodon.org/tags/pandas" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#pandas</a> <a href="https://fosstodon.org/tags/polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#polars</a>

Data SciencePolars is a lightning fast DataFrame library/in-memory query engine with parallel execution and cache efficiency. And now you can use is with the tidyverse syntax: <a href="https://www.tidypolars.etiennebacher.com/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://www.tidypolars.etiennebacher.com/</a> <a href="https://genomic.social/tags/rstats" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#rstats</a> <a href="https://genomic.social/tags/polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#polars</a> <a href="https://genomic.social/tags/optimisation" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#optimisation</a>

Mainmatter🌊  Kisters provides real-time flood warnings—but traffic spikes made scaling costly. We helped them build a Rust-based system using Polars to process data efficiently across servers, edge, and browsers (WASM)—keeping performance high and costs low.Read more ➡️ <a href="https://mainmatter.com/cases/kisters/?utm_source=linkedin" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://mainmatter.com/cases/kisters/?utm_source=linkedin</a> <a href="https://fosstodon.org/tags/rustlang" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#rustlang</a> <a href="https://fosstodon.org/tags/WASM" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#WASM</a> <a href="https://fosstodon.org/tags/edgecomputing" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#edgecomputing</a> <a href="https://fosstodon.org/tags/cloudoptimization" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#cloudoptimization</a> <a href="https://fosstodon.org/tags/polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#polars</a>

WolfAt last night’s <a href="https://hachyderm.io/@mug" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@mug</a> meeting we looked at a lot of different solutions to <a href="https://hachyderm.io/tags/adventofcode" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#adventofcode</a> day 1 in many different languages. Two that were very interesting to me were <a href="https://hachyderm.io/tags/Zig" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Zig</a> and <a href="https://hachyderm.io/tags/haskell" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#haskell</a>. The way these two languages worked was really quite fascinating. After seeing real code in these two languages, I can tell they are not for me; but they were interesting and illuminating nonetheless. There was a solution entirely in <a href="https://hachyderm.io/tags/SQL" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SQL</a>. Another in <a href="https://hachyderm.io/tags/vim9script" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#vim9script</a>. Another in <a href="https://hachyderm.io/tags/swiftlang" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#swiftlang</a> <a href="https://hachyderm.io/tags/swift" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#swift</a> (I don’t think that one’s in the repo yet). I wrote several implementations myself. The one I felt most proud of is <a href="https://hachyderm.io/tags/Python" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Python</a> with the core written in <a href="https://hachyderm.io/tags/rustlang" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#rustlang</a> <a href="https://hachyderm.io/tags/rust" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#rust</a> tied together with <a href="https://hachyderm.io/tags/PyO3" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#PyO3</a>. The one I felt was maybe the best tool for the job was entirely based on <a href="https://hachyderm.io/tags/pandas" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#pandas</a>. As I said in a previous post, I tried to solve it in <a href="https://hachyderm.io/tags/polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#polars</a>, but the API exposed by Polars at least as far as I could tell, made it no better than simple lists in Python. I need to get deeper knowledge here. The repo lives here: <a href="https://github.com/MichiganUnixUserGroup/MUG-2025-03-11-Advent-of-Code" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://github.com/MichiganUnixUserGroup/MUG-2025-03-11-Advent-of-Code</a>.

WolfI solved a small problem trying to do it using only <a href="https://hachyderm.io/tags/pandas" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#pandas</a> (it was <a href="https://hachyderm.io/tags/adventofcode" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#adventofcode</a> day 1). I know Pandas pretty well. The solution to both parts together turned out to be very short. Tried to do the same thing in <a href="https://hachyderm.io/tags/polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#polars</a>. Way harder. Didn’t get to leverage DataFrames. Basically just used Series and they were just glorified lists. This was my first experience with Polars. So my ignorance was certainly a factor. But wow. Pandas just did the job better. I’m sure Polars is still growing. I really want to like it!

Eric R. ScottThere are now so many "backends" for {dplyr}—duckdb with {duckplyr}, polars with {tidypolars}, various database engines with {dbplyr}, {data.table} with {dtplyr}. Is there a blog post or flow chart somewhere with pros and cons of each? Like, comparisons of memory requirements, speed, and how likely they are to "just work"?<a href="https://fosstodon.org/tags/rstats" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#rstats</a> <a href="https://fosstodon.org/tags/duckdb" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#duckdb</a> <a href="https://fosstodon.org/tags/dplyr" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#dplyr</a> <a href="https://fosstodon.org/tags/tidyverse" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#tidyverse</a> <a href="https://fosstodon.org/tags/polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#polars</a>

N-gated Hacker NewsAh, nothing screams "cutting-edge innovation" like using <a href="https://mastodon.social/tags/Parquet" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Parquet</a> and <a href="https://mastodon.social/tags/Polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Polars</a> for text embeddings. 🤣 Because, clearly, what the AI world needed was some spreadsheet nostalgia. And don't forget, everyone desperately needed to know how to embed 32,254 Magic the Gathering cards. 🧙‍♂️💾 Truly groundbreaking stuff! <a href="https://minimaxir.com/2025/02/embeddings-parquet/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://minimaxir.com/2025/02/embeddings-parquet/</a> <a href="https://mastodon.social/tags/cuttingedgeinnovation" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#cuttingedgeinnovation</a> <a href="https://mastodon.social/tags/textembeddings" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#textembeddings</a> <a href="https://mastodon.social/tags/MagicTheGathering" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#MagicTheGathering</a> <a href="https://mastodon.social/tags/AIhumor" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AIhumor</a> <a href="https://mastodon.social/tags/HackerNews" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#HackerNews</a> <a href="https://mastodon.social/tags/ngated" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ngated</a>

Hacker NewsThe Best Way to Use Text Embeddings Portably Is with Parquet and Polars — <a href="https://minimaxir.com/2025/02/embeddings-parquet/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://minimaxir.com/2025/02/embeddings-parquet/</a> <a href="https://mastodon.social/tags/HackerNews" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#HackerNews</a> <a href="https://mastodon.social/tags/TextEmbeddings" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#TextEmbeddings</a> <a href="https://mastodon.social/tags/Parquet" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Parquet</a> <a href="https://mastodon.social/tags/Polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Polars</a> <a href="https://mastodon.social/tags/DataScience" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#DataScience</a> <a href="https://mastodon.social/tags/MachineLearning" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#MachineLearning</a>

benrutterWas noodling around with Gleam a while back, needed a dataframe library and set up a half-baked interface for polars.I can't beleive how easy the js ffi made it! Language FFI is such a hidden superpower, more of this please from bew languages! 😍<a href="https://mastodon.green/tags/programming" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#programming</a> <a href="https://mastodon.green/tags/gleam" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#gleam</a> <a href="https://mastodon.green/tags/polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#polars</a>

Alexandre B A Villares 🐍"Plotando estatísticas básicas com <a href="https://ciberlandia.pt/tags/Polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Polars</a> e <a href="https://ciberlandia.pt/tags/Matplotlib" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Matplotlib</a> - <a href="https://ciberlandia.pt/tags/NLP" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#NLP</a> 04 " <a href="https://bolha.us/@dunossauro" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@dunossauro</a> <a href="https://ciberlandia.pt/tags/Python" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Python</a><a href="https://www.youtube.com/watch?v=4HpSFIekqDw" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://www.youtube.com/watch?v=4HpSFIekqDw</a> Hoje eu aprendi uma ideia ótima do Dunossauro que é imaginar que o ax do fig, ax do matplotlib (axis/eixo) é como uma haste onde penduramos as coisas! Como é fundamental o trabalho dele pra nossa comunidade.Update: inicialmente achei que era uma tradução corrente mas ele me explicou que não.

Jim GardnerDoes anyone have advice for unit testing a Polars Python expression? Say I have:def my_func(a, b): return (pl.col(a) / pl.col(b))How do you unit test? We want to compare the result of this function to an expectation. At the moment we’re wrapping the result in str(), which gives us a stringified version of the expression. Then we assert that it is equal to the stringified expression that we would expect.It works, just feels a little bit fragile. Unfortunately you can’t compare expressions directly with the equality operator ==. And polars.testing doesn’t have asserts for it either.(We are of course also going to have integration tests, where the expression actually gets used on a df fixture)<a href="https://bird.makeup/users/jeroenhjanssens" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@jeroenhjanssens</a> <a href="https://hachyderm.io/tags/Polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Polars</a> <a href="https://hachyderm.io/tags/Python" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Python</a> <a href="https://hachyderm.io/tags/PyData" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#PyData</a>

Isa BernardiniUne auteure de <a href="https://mastodon.social/tags/polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#polars</a> à suivre assurément. J’en ai lu plusieurs d’elle, dont « Block 46 ». Voici son « dernier » sorti. Je ne gagne absolument rien à faire sa pub, je suis juste une personne qui aime ce qu’elle <a href="https://mastodon.social/tags/%C3%A9crit" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#écrit</a> et représente. Une bien belle personne.

Claudine Layre<a href="https://maisonjuliengracq.fr/evenement/affaire-classee-noir-sur-loire/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://maisonjuliengracq.fr/evenement/affaire-classee-noir-sur-loire/</a> <a href="https://mastodon.social/tags/culture" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#culture</a> <a href="https://mastodon.social/tags/litt%C3%A9rature" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#littérature</a> <a href="https://mastodon.social/tags/polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#polars</a>

Markus SuhrJust caught up with the recent Delta Lake webinar, > Revolutionizing Delta Lake workflows on AWS Lambda with Polars, DuckDB, Daft & RustSome interesting hints there regarding lightweight processing of big-ish data. Easy to relate to any other framework instead of Lambda, e.g. <a href="https://gruene.social/tags/ApacheAirflow" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ApacheAirflow</a> tasks<a href="https://youtu.be/BR9oFD0QMAs" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://youtu.be/BR9oFD0QMAs</a><a href="https://gruene.social/tags/dataengineering" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#dataengineering</a> <a href="https://gruene.social/tags/datascience" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#datascience</a> <a href="https://gruene.social/tags/duckdb" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#duckdb</a> <a href="https://gruene.social/tags/daft" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#daft</a> <a href="https://gruene.social/tags/polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#polars</a> <a href="https://gruene.social/tags/pandas" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#pandas</a> <a href="https://gruene.social/tags/python" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#python</a> <a href="https://gruene.social/tags/spark" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#spark</a> <a href="https://gruene.social/tags/deltalake" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#deltalake</a> <a href="https://gruene.social/tags/databricks" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#databricks</a> <a href="https://gruene.social/tags/airflow" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#airflow</a> <a href="https://gruene.social/tags/bigdata" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#bigdata</a> <a href="https://gruene.social/tags/smalldata" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#smalldata</a>

Alexandre B A Villares 🐍"Estáticas básicas de texto com <a href="https://ciberlandia.pt/tags/Polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Polars</a> e <a href="https://ciberlandia.pt/tags/Spacy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Spacy</a> - <a href="https://ciberlandia.pt/tags/NLP" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#NLP</a> 03 " <a href="https://ciberlandia.pt/tags/Python" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Python</a> <a href="https://www.youtube.com/watch?v=FgsdmxILSFo" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://www.youtube.com/watch?v=FgsdmxILSFo</a>

WolfLots of <a href="https://hachyderm.io/tags/rustlang" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#rustlang</a> over the weekend. Lots of stuff to say. I’m writing small programs (mostly <a href="https://hachyderm.io/tags/adventofcode" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#adventofcode</a>) with lots of looking stuff up. Looking stuff up shows me just how much _more_ Rust there is to know. Rust chained iterator expressions can do everything a <a href="https://hachyderm.io/tags/Python" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Python</a> list <a href="https://hachyderm.io/tags/comprehension" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#comprehension</a> can do. I use <a href="https://hachyderm.io/tags/pandas" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#pandas</a> all the time. <a href="https://hachyderm.io/tags/polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#polars</a> is data compatible and you can call it from Rust. This could help me in my job. The way you implement methods in Rust is _so_ much like type-bound procedures in <a href="https://hachyderm.io/tags/oberon2" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#oberon2</a>. I have more and more respect for Rust. I still love Python.

Recent searches

Search options

Administered by:

Server stats:

#polars