You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm waiting for the arrow-avro PR below as it might include changes to the public API. apache/arrow-rs#6965
I will resume related work once this PR is merged and arrow-avro is released.
Is your feature request related to a problem or challenge?
Currently DataFusion has its own avro --> arrow implementation in: https://github.com/apache/datafusion/blob/54a5d3fd3f98997f048c2e3e7ba67d7bfe2f124d/datafusion/core/src/datasource/avro_to_arrow/mod.rs
This was added way back in the day by @Igosuki and @jorgecarleitao in
Recently there is work upstream in arrow-rs to add additional avro support:
For example, the recent PR from @jecsand838
Describe the solution you'd like
One arrow-avro is more mature, it would be great to consider removing the DataFusion specific implementation and using the upstream arrow-avro one
This would allow DataFusion to take advantage of the upstream work and keep our codebase a bit simpler
Describe alternatives you've considered
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: