Create accountLogin

Recent searches

No recent searches

Search options

Only available when logged in.
techhub.social is one of the many independent Mastodon servers you can use to participate in the fediverse.
A hub primarily for passionate technologists, but everyone is welcome

Administered by:

Server stats:

5.4K
active users

techhub.social: AboutStatusProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.3.6

#DataEngineering

9 posts8 participants0 posts today
Olivier D'Hondt 🛰️🌍🌱<p><a href="https://framapiaf.org/tags/dask" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>dask</span></a> is strange. Sometimes using the dask counterpart to numpy functions or arrays makes computations slower. Sometimes not. Also, lots of variability in runtime. <a href="https://framapiaf.org/tags/python" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>python</span></a> <a href="https://framapiaf.org/tags/dataengineering" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>dataengineering</span></a></p>
pipTrends<p>We all know that SELECT * FROM table_name is less efficient than selecting specific columns. The same applies to Django ORM, where you can optimize application performance using defer() and only(). If you want to learn how to use them, check out this article by Nik Tomazic.</p><p><a href="https://testdriven.io/blog/django-query-optimization/" rel="nofollow noopener noreferrer" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">testdriven.io/blog/django-quer</span><span class="invisible">y-optimization/</span></a></p><p><a href="https://mastodon.social/tags/python" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>python</span></a> <a href="https://mastodon.social/tags/Programming" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Programming</span></a> <a href="https://mastodon.social/tags/PythonProgramming" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>PythonProgramming</span></a> <a href="https://mastodon.social/tags/ai" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ai</span></a> <a href="https://mastodon.social/tags/ml" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ml</span></a> <a href="https://mastodon.social/tags/MachineLearning" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>MachineLearning</span></a> <a href="https://mastodon.social/tags/SoftwareDevelopment" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>SoftwareDevelopment</span></a> <a href="https://mastodon.social/tags/WebDevelopment" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>WebDevelopment</span></a> <a href="https://mastodon.social/tags/TechNews" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>TechNews</span></a> <a href="https://mastodon.social/tags/OpenSource" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>OpenSource</span></a> <a href="https://mastodon.social/tags/DataEngineering" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>DataEngineering</span></a></p>
SchifferMarget<p>&quot;Supercharge ETL pipelines with ZillizAI&#39;s Milvus! Extract, transform, load data at scale, lightning-fast. Unlock next-gen efficiency. 🔥 <a href="https://techhub.social/tags/DataEngineering" class="mention hashtag" rel="tag">#<span>DataEngineering</span></a> <a href="https://techhub.social/tags/BigData" class="mention hashtag" rel="tag">#<span>BigData</span></a> <a href="https://techhub.social/tags/VectorDatabase" class="mention hashtag" rel="tag">#<span>VectorDatabase</span></a>&quot; <a href="https://zilliz.com/glossary/etl" target="_blank" rel="nofollow noopener noreferrer" translate="no"><span class="invisible">https://</span><span class="">zilliz.com/glossary/etl</span><span class="invisible"></span></a></p>
SchifferMarget<p>Boost DB performance with denormalization! Reduce joins, speed queries, handle redundancy. Supercharge vector searches at scale with Zilliz or Milvus. <a href="https://techhub.social/tags/DataEngineering" class="mention hashtag" rel="tag">#<span>DataEngineering</span></a> <a href="https://zilliz.com/glossary/database-denormalization" target="_blank" rel="nofollow noopener noreferrer" translate="no"><span class="invisible">https://</span><span class="ellipsis">zilliz.com/glossary/database-d</span><span class="invisible">enormalization</span></a></p>
siva143<p>Visualpath offers expert-led Top GCP Data Engineer Training in Hyderabad with a hands-on learning approach. Our Google Data Engineer Certification is available in Hyderabad, Chennai, Bangalore, and worldwide, allowing you to learn from anywhere. Designed by industry professionals, this program provides real-world expertise to enhance your skills. For more details, contact +91-7032290546<br>Visit: <a href="https://www.visualpath.in/gcp-data-engineer-online-training.html" rel="nofollow noopener noreferrer" translate="no" target="_blank"><span class="invisible">https://www.</span><span class="ellipsis">visualpath.in/gcp-data-enginee</span><span class="invisible">r-online-training.html</span></a> </p><p><a href="https://mastodon.social/tags/Visualpath" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Visualpath</span></a> <a href="https://mastodon.social/tags/GCP" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>GCP</span></a> <a href="https://mastodon.social/tags/DataEngineer" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>DataEngineer</span></a> <a href="https://mastodon.social/tags/dataengineering" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>dataengineering</span></a> <a href="https://mastodon.social/tags/GoogleCloud" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>GoogleCloud</span></a></p>
pipTrends<p>Hot-reloading in Python can be slow since the entire server process must be restarted from scratch. The Tach package speeds this up by building a dependency map. Evan Doyle explained how it works with an interesting example.</p><p><a href="https://www.gauge.sh/blog/how-to-build-hot-module-replacement-in-python" rel="nofollow noopener noreferrer" translate="no" target="_blank"><span class="invisible">https://www.</span><span class="ellipsis">gauge.sh/blog/how-to-build-hot</span><span class="invisible">-module-replacement-in-python</span></a></p><p><a href="https://mastodon.social/tags/python" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>python</span></a> <a href="https://mastodon.social/tags/Programming" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Programming</span></a> <a href="https://mastodon.social/tags/PythonProgramming" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>PythonProgramming</span></a> <a href="https://mastodon.social/tags/ai" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ai</span></a> <a href="https://mastodon.social/tags/ml" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ml</span></a> <a href="https://mastodon.social/tags/MachineLearning" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>MachineLearning</span></a> <a href="https://mastodon.social/tags/SoftwareDevelopment" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>SoftwareDevelopment</span></a> <a href="https://mastodon.social/tags/WebDevelopment" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>WebDevelopment</span></a> <a href="https://mastodon.social/tags/TechNews" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>TechNews</span></a> <a href="https://mastodon.social/tags/OpenSource" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>OpenSource</span></a> <a href="https://mastodon.social/tags/DataEngineering" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>DataEngineering</span></a></p>
Hacker News<p>LLM Workflows then Agents: Getting Started with Apache Airflow</p><p><a href="https://github.com/astronomer/airflow-ai-sdk" rel="nofollow noopener noreferrer" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">github.com/astronomer/airflow-</span><span class="invisible">ai-sdk</span></a></p><p><a href="https://mastodon.social/tags/HackerNews" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>HackerNews</span></a> <a href="https://mastodon.social/tags/LLM" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>LLM</span></a> <a href="https://mastodon.social/tags/Workflows" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Workflows</span></a> <a href="https://mastodon.social/tags/Agents" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Agents</span></a> <a href="https://mastodon.social/tags/ApacheAirflow" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ApacheAirflow</span></a> <a href="https://mastodon.social/tags/DataEngineering" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>DataEngineering</span></a> <a href="https://mastodon.social/tags/AIWorkflow" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>AIWorkflow</span></a></p>
Ithar Malik<p>Excited to share our first steps in implementing <a href="https://mastodon.social/tags/DataMesh" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>DataMesh</span></a> at scale! 🚀 From strategy to tech, operating models to culture—this paradigm shift is challenging but transformative. Full read: <a href="https://itharmalik.com/2025/03/30/embarking-on-our-data-mesh-journey-the-first-steps/" rel="nofollow noopener noreferrer" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">itharmalik.com/2025/03/30/emba</span><span class="invisible">rking-on-our-data-mesh-journey-the-first-steps/</span></a> <a href="https://mastodon.social/tags/DataEngineering" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>DataEngineering</span></a> <a href="https://mastodon.social/tags/Innovation" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Innovation</span></a></p>
zartom<p>Troubleshooting Airflow SQL Server Errors: A Case Study of System Task Initialization Failure<br>Troubleshoot frustrating Airflow SQL Server errors! This post helps diagnose &amp; fix system task initialization failures and other connection issues. Learn to build robust data pipelines. <a href="https://mastodon.social/tags/AirflowSQLServerErrors" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>AirflowSQLServerErrors</span></a> <a href="https://mastodon.social/tags/DataEngineering" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>DataEngineering</span></a> <a href="https://mastodon.social/tags/SQLServer" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>SQLServer</span></a> <a href="https://mastodon.social/tags/ApacheAirflow" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ApacheAirflow</span></a> <a href="https://mastodon.social/tags/DatabaseConnection" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>DatabaseConnection</span></a> <a href="https://mastodon.social/tags/DataPipelines" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>DataPipelines</span></a><br><a href="https://tech-champion.com/database/sql-server/troubleshooting-airflow-sql-server-errors-a-case-study-of-system-tas" rel="nofollow noopener noreferrer" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">tech-champion.com/database/sql</span><span class="invisible">-server/troubleshooting-airflow-sql-server-errors-a-case-study-of-system-tas</span></a>...</p>
Will Hopkins 🌈📸<p><a href="https://a2mi.social/tags/dataengineering" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>dataengineering</span></a> question for y'all: have you used GPU-accellerated Spark for **non-ML/LLM/AI workloads**? If so, what was your use case, where did you host it, and how did you find it performed against standard Spark for your use case?</p><p>Just curious about broad strokes.</p>
pipTrends<p>This week newsletter will be out tomorrow. Interesting stuff by Tanner Young, Max Gabrielsson, Tom Nijhof-Verhees, Charles de Villiers &amp; Lee Vaughan covered</p><p><a href="https://newsletter.piptrends.com/p/async-agnostic-decorators-vector" rel="nofollow noopener noreferrer" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">newsletter.piptrends.com/p/asy</span><span class="invisible">nc-agnostic-decorators-vector</span></a></p><p><a href="https://mastodon.social/tags/python" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>python</span></a> <a href="https://mastodon.social/tags/Programming" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Programming</span></a> <a href="https://mastodon.social/tags/PythonProgramming" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>PythonProgramming</span></a> <a href="https://mastodon.social/tags/ai" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ai</span></a> <a href="https://mastodon.social/tags/ml" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ml</span></a> <a href="https://mastodon.social/tags/SoftwareDevelopment" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>SoftwareDevelopment</span></a> <a href="https://mastodon.social/tags/TechNews" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>TechNews</span></a> <a href="https://mastodon.social/tags/OpenSource" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>OpenSource</span></a> <a href="https://mastodon.social/tags/DataEngineering" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>DataEngineering</span></a></p>
pipTrends<p>Want to build amazing UIs in the terminal? Textual is one of the best options out there! If you're interested in learning how to use it, check out this fantastic <span class="h-card" translate="no"><a href="https://fosstodon.org/@realpython" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@<span>realpython</span></a></span> article by Charles de Villiers.</p><p><a href="https://realpython.com/python-textual/" rel="nofollow noopener noreferrer" translate="no" target="_blank"><span class="invisible">https://</span><span class="">realpython.com/python-textual/</span><span class="invisible"></span></a></p><p><a href="https://mastodon.social/tags/python" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>python</span></a> <a href="https://mastodon.social/tags/Programming" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Programming</span></a> <a href="https://mastodon.social/tags/PythonProgramming" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>PythonProgramming</span></a> <a href="https://mastodon.social/tags/DataScience" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>DataScience</span></a> <a href="https://mastodon.social/tags/MachineLearning" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>MachineLearning</span></a> <a href="https://mastodon.social/tags/SoftwareDevelopment" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>SoftwareDevelopment</span></a> <a href="https://mastodon.social/tags/WebDevelopment" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>WebDevelopment</span></a> <a href="https://mastodon.social/tags/TechNews" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>TechNews</span></a> <a href="https://mastodon.social/tags/OpenSource" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>OpenSource</span></a> <a href="https://mastodon.social/tags/DataEngineering" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>DataEngineering</span></a></p>
pipTrends<p>If you don’t want to set up a separate database server for vector search, you can use SQLite or DuckDB with vector extension. In this article, Max Gabrielsson explained how to do it using DuckDB.</p><p><a href="https://duckdb.org/2024/05/03/vector-similarity-search-vss.html" rel="nofollow noopener noreferrer" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">duckdb.org/2024/05/03/vector-s</span><span class="invisible">imilarity-search-vss.html</span></a></p><p><a href="https://mastodon.social/tags/python" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>python</span></a> <a href="https://mastodon.social/tags/Programming" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Programming</span></a> <a href="https://mastodon.social/tags/PythonProgramming" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>PythonProgramming</span></a> <a href="https://mastodon.social/tags/DataScience" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>DataScience</span></a> <a href="https://mastodon.social/tags/MachineLearning" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>MachineLearning</span></a> <a href="https://mastodon.social/tags/SoftwareDevelopment" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>SoftwareDevelopment</span></a> <a href="https://mastodon.social/tags/WebDevelopment" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>WebDevelopment</span></a> <a href="https://mastodon.social/tags/TechNews" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>TechNews</span></a> <a href="https://mastodon.social/tags/OpenSource" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>OpenSource</span></a> <a href="https://mastodon.social/tags/DataEngineering" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>DataEngineering</span></a></p>
N-gated Hacker News<p>🚀🎉 Oh joy, another techie proudly smashes <a href="https://mastodon.social/tags/pgvector" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>pgvector</span></a> into tiny pieces like a toddler with a LEGO set, all in the name of "optimization."🙄 Apparently, splitting tables between machines is "easier"—just like juggling chainsaws! 🔥💻<br><a href="https://pgdog.dev/blog/sharding-pgvector" rel="nofollow noopener noreferrer" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">pgdog.dev/blog/sharding-pgvect</span><span class="invisible">or</span></a> <a href="https://mastodon.social/tags/technews" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>technews</span></a> <a href="https://mastodon.social/tags/optimization" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>optimization</span></a> <a href="https://mastodon.social/tags/dataengineering" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>dataengineering</span></a> <a href="https://mastodon.social/tags/jugglingchainsaws" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>jugglingchainsaws</span></a> <a href="https://mastodon.social/tags/HackerNews" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>HackerNews</span></a> <a href="https://mastodon.social/tags/ngated" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ngated</span></a></p>
GRASS GIS<p>Let us know what topics you'd like to see covered as a GRASS tutorial!</p><p>Comment your suggestions below👇</p><p><a href="https://grass-tutorial.osgeo.org" rel="nofollow noopener noreferrer" translate="no" target="_blank"><span class="invisible">https://</span><span class="">grass-tutorial.osgeo.org</span><span class="invisible"></span></a></p><p><a href="https://fosstodon.org/tags/foss" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>foss</span></a> <a href="https://fosstodon.org/tags/osgeo" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>osgeo</span></a> <a href="https://fosstodon.org/tags/geospatial" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>geospatial</span></a> <a href="https://fosstodon.org/tags/grassgis" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>grassgis</span></a> <a href="https://fosstodon.org/tags/datascience" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>datascience</span></a> <a href="https://fosstodon.org/tags/python" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>python</span></a> <a href="https://fosstodon.org/tags/r" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>r</span></a> <a href="https://fosstodon.org/tags/remotesensing" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>remotesensing</span></a> <a href="https://fosstodon.org/tags/hydrology" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>hydrology</span></a> <a href="https://fosstodon.org/tags/dataengineering" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>dataengineering</span></a><br><a href="https://github.com/OSGeo/grass-tutorials" rel="nofollow noopener noreferrer" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">github.com/OSGeo/grass-tutoria</span><span class="invisible">ls</span></a></p>
Doug Ortiz<p>🚀 Course update: "Leveraging PostgreSQL with RAG" chapter now available! </p><p>Discover how to: <br>- Integrate vector databases with LLMs <br>- Implement retrieval-augmented generation patterns <br>- Optimize SQL workflows using @vanna_ai </p><p>Perfect for developers building AI-enhanced data systems. Includes new video walkthrough: <a href="https://link.illustris.org/intro2vannaai" rel="nofollow noopener noreferrer" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">link.illustris.org/intro2vanna</span><span class="invisible">ai</span></a> </p><p><a href="https://mastodon.social/tags/PostgreSQL" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>PostgreSQL</span></a> <a href="https://mastodon.social/tags/AIDevelopment" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>AIDevelopment</span></a> <a href="https://mastodon.social/tags/DataEngineering" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>DataEngineering</span></a></p>
Vic<p>Data engineering teams in Austria face several challenges, including upstream data quality issues and the lack of clear data ownership. These obstacles hinder the efficient delivery of quality data to consumers.</p><p>Pain Points:<br />🔹Upstream data quality issues.<br />🔹Lack of data ownership and governance.<br />🔹Inconsistent data collection and processing.</p><p>As highlighted in Reddit discussions, upstream data quality issues are a significant challenge. Additionally, Austrian data analytics challenges include privacy, security, and data ownership concerns.</p><p>Reddit discussions - <a href="https://www.reddit.com/r/dataengineering/comments/17so76t/what_are_the_biggest_obstaclespainpoints_in_data/?rdt=47169" target="_blank" rel="nofollow noopener noreferrer" translate="no"><span class="invisible">https://www.</span><span class="ellipsis">reddit.com/r/dataengineering/c</span><span class="invisible">omments/17so76t/what_are_the_biggest_obstaclespainpoints_in_data/?rdt=47169</span></a></p><p>Austrian data analytics challenges - <a href="https://iktderzukunft.at/resources/pdf/conquering-data-in-austria.pdf" target="_blank" rel="nofollow noopener noreferrer" translate="no"><span class="invisible">https://</span><span class="ellipsis">iktderzukunft.at/resources/pdf</span><span class="invisible">/conquering-data-in-austria.pdf</span></a></p><p><a href="https://techhub.social/tags/DataEngineering" class="mention hashtag" rel="tag">#<span>DataEngineering</span></a> <a href="https://techhub.social/tags/Vienna" class="mention hashtag" rel="tag">#<span>Vienna</span></a> <a href="https://techhub.social/tags/TechHive" class="mention hashtag" rel="tag">#<span>TechHive</span></a> <a href="https://techhub.social/tags/StaffingSolutions" class="mention hashtag" rel="tag">#<span>StaffingSolutions</span></a> <a href="https://techhub.social/tags/Data" class="mention hashtag" rel="tag">#<span>Data</span></a> <a href="https://techhub.social/tags/Austria" class="mention hashtag" rel="tag">#<span>Austria</span></a></p>
LavX News<p>Revolutionizing Data Pipelines: Apache Iceberg Streaming Writes and Batch Reads in C++</p><p>The latest PR from Timeplus introduces a groundbreaking MVP-level support for Apache Iceberg, enabling seamless streaming writes and batch reads directly in C++. This development not only enhances dat...</p><p><a href="https://news.lavx.hu/article/revolutionizing-data-pipelines-apache-iceberg-streaming-writes-and-batch-reads-in-c" rel="nofollow noopener noreferrer" target="_blank"><span class="invisible">https://</span><span class="ellipsis">news.lavx.hu/article/revolutio</span><span class="invisible">nizing-data-pipelines-apache-iceberg-streaming-writes-and-batch-reads-in-c</span></a></p><p><a href="https://mastodon.cloud/tags/news" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>news</span></a> <a href="https://mastodon.cloud/tags/tech" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>tech</span></a> <a href="https://mastodon.cloud/tags/CPlusPlus" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>CPlusPlus</span></a> <a href="https://mastodon.cloud/tags/DataEngineering" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>DataEngineering</span></a> <a href="https://mastodon.cloud/tags/ApacheIceberg" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ApacheIceberg</span></a></p>
emmuzoo<p>Mission complete! ✅ Just finished the <a href="https://mastodon.social/tags/DataEngineering" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>DataEngineering</span></a> homework: identifying the longest uninterrupted streak of taxi rides in a 5-minute window using <a href="https://mastodon.social/tags/Kafka" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Kafka</span></a> and <a href="https://mastodon.social/tags/Flink" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Flink</span></a>. Feeling proud of the progress so far! 🚖</p><p><a href="https://mastodon.social/tags/DataTalksClub" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>DataTalksClub</span></a> <a href="https://mastodon.social/tags/Streaming" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Streaming</span></a> <a href="https://mastodon.social/tags/Kafka" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Kafka</span></a> <a href="https://mastodon.social/tags/Flink" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Flink</span></a> <a href="https://mastodon.social/tags/PyFlink" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>PyFlink</span></a> <a href="https://mastodon.social/tags/DataScience" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>DataScience</span></a> <a href="https://mastodon.social/tags/TaxiNY" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>TaxiNY</span></a></p>
emmuzoo<p>Wrapping up Module 6 of <a href="https://mastodon.social/tags/DataEngineering" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>DataEngineering</span></a> and diving into the homework! 📝 The task: find the longest uninterrupted streak of taxi rides in a 5-minute window using <a href="https://mastodon.social/tags/Kafka" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Kafka</span></a> and <a href="https://mastodon.social/tags/Flink" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Flink</span></a>. Challenge accepted! 🚖💨</p><p><a href="https://mastodon.social/tags/DataTalksClub" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>DataTalksClub</span></a> <a href="https://mastodon.social/tags/Streaming" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Streaming</span></a> <a href="https://mastodon.social/tags/Kafka" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Kafka</span></a> <a href="https://mastodon.social/tags/Flink" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Flink</span></a> <a href="https://mastodon.social/tags/PyFlink" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>PyFlink</span></a> <a href="https://mastodon.social/tags/DataScience" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>DataScience</span></a></p>
ExploreLive feeds
About