feat(python): expose remaining Topic/TopicDetails fields and partitions by mattp5657 · Pull Request #3623 · apache/iggy

mattp5657 · 2026-07-07T00:01:29Z

Which issue does this PR address?

Rationale

Continuation of #3572. That PR exposed Topic/TopicDetails basics but left several fields returned by the server unavailable in Python, so callers couldn't inspect a topic's expiry, size, max size, or its partitions without another SDK.

What changed?

Topic and TopicDetails previously exposed only id, name, partitions_count, compression_algorithm, and replication_factor, leaving created_at, size, message_expiry, max_topic_size, and per-partition data unavailable in Python.

Both classes now expose created_at, size, message_expiry (new IggyExpiry enum), and max_topic_size (new MaxTopicSize enum). TopicDetails also gains partitions, a list of new Partition objects. Partition (core/common) now derives Clone to support building that list.

Local Execution

Passed
Pre-commit hooks ran

AI Usage

Claude was used to help generate and review this PR.

codecov · 2026-07-07T00:09:16Z

Codecov Report

❌ Patch coverage is 83.22981% with 27 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.43%. Comparing base (354be5c) to head (a511223).
⚠️ Report is 41 commits behind head on master.

Files with missing lines	Patch %	Lines
foreign/python/src/topic.rs	78.18%	24 Missing ⚠️
foreign/python/src/consumer.rs	87.50%	2 Missing ⚠️
foreign/python/src/client.rs	96.87%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #3623      +/-   ##
============================================
- Coverage     74.11%   71.43%   -2.68%     
  Complexity      937      937              
============================================
  Files          1262     1286      +24     
  Lines        132757   136080    +3323     
  Branches     108647   111880    +3233     
============================================
- Hits          98396    97213    -1183     
- Misses        31273    35360    +4087     
- Partials       3088     3507     +419

Components	Coverage Δ
Rust Core	`71.24% <ø> (-3.28%)`	⬇️
Java SDK	`62.44% <ø> (ø)`
C# SDK	`71.40% <ø> (-0.71%)`	⬇️
Python SDK	`90.36% <83.22%> (-1.30%)`	⬇️
PHP SDK	`84.29% <ø> (ø)`
Node SDK	`91.35% <ø> (ø)`
Go SDK	`42.28% <ø> (ø)`

Files with missing lines	Coverage Δ
foreign/python/src/lib.rs	`100.00% <100.00%> (ø)`
foreign/python/src/client.rs	`98.90% <96.87%> (+0.02%)`	⬆️
foreign/python/src/consumer.rs	`81.11% <87.50%> (+0.57%)`	⬆️
foreign/python/src/topic.rs	`82.66% <78.18%> (-12.34%)`	⬇️

... and 239 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

slbotbm

You've defined MaxTopicSize and IggyExpiry as enums here, but the sdk utilizes normal python ints and timedeltas. Let's change those functions to accept your enums instead of the python ints and structs.

slbotbm

will review in more depth later when I get more time

slbotbm · 2026-07-08T06:15:06Z

+    /// Use the server's default message expiry.
+    ServerDefault(),
+    /// Messages expire after this duration.
+    ExpireDuration { duration: Py<PyDelta> },
+    /// Messages never expire.
+    NeverExpire(),


Please also write exactly what does server default and never expire mean. Also, range of values that expireduration accepts

slbotbm · 2026-07-08T06:16:16Z

+pub enum MaxTopicSize {
+    /// Use the server's default max size.
+    ServerDefault(),
+    /// The topic is limited to this many bytes.
+    Custom { bytes: u64 },
+    /// The topic has no maximum size.
+    Unlimited(),
+}


Same here. Make the docs explicit. We are doing this since the user cannot trace what exactly these mean without looking at rust sdk code.

slbotbm · 2026-07-09T14:12:17Z

/author

mattp5657 · 2026-07-10T12:31:04Z

You've defined MaxTopicSize and IggyExpiry as enums here, but the sdk utilizes normal python ints and timedeltas. Let's change those functions to accept your enums instead of the python ints and structs.

This was an oversight from me, thanks for the call out :)

…d update_topic

mattp5657 · 2026-07-10T23:38:40Z

I made the outlined changes, my only concern is if we should continue with backwards compatibility and incorporate it into testing. But wanted to get your thoughts before doing so. :)

mattp5657 · 2026-07-10T23:39:09Z

/ready

slbotbm

Some comments

slbotbm · 2026-07-12T07:32:49Z

+        match expiry {
+            IggyExpiry::ServerDefault() => RustIggyExpiry::ServerDefault,
+            IggyExpiry::ExpireDuration { duration } => {
+                RustIggyExpiry::ExpireDuration(py_delta_to_iggy_duration(duration))


This also accepts negative duration, which should be rejected as PyValueError

slbotbm · 2026-07-12T07:33:35Z

+    /// the server deletes the oldest sealed segments to make room for new
+    /// messages.
+    ///
+    /// `bytes` must be greater than zero and less than `u64::MAX`: those two


Python callers do not know what u64::MAX means.

mattp5657 · 2026-07-12T23:38:06Z

Fixed both boundary-collision issues flagged in review: negative timedelta values for IggyExpiry.ExpireDuration and 0/u64::MAX values for MaxTopicSize.Custom were silently accepted and misinterpreted on the wire (as ServerDefault/Unlimited) instead of being rejected. Both conversions now raise PyValueError for out-of-range input, propagated through create_topic/update_topic. Updated the affected doc comments/stubs and added a regression test for each invalid case.

mattp5657 · 2026-07-12T23:38:30Z

/ready

hubcio · 2026-07-15T09:34:46Z

    Python::attach(|py| {
        let delta = delta1.bind(py);
-        let seconds = (delta.get_days() * 60 * 60 * 24 + delta.get_seconds()) as u64;
+        let total_seconds = delta.get_days() * 60 * 60 * 24 + delta.get_seconds();


delta.get_days() * 60 * 60 * 24 + delta.get_seconds() is all i32 here (pyo3's get_days/get_seconds both return i32). for |days| >= 24856 (~68yr) the multiply overflows - release wheels wrap silently, debug builds panic. the nasty part is it defeats the negative-duration guard this PR just added right below: a large negative timedelta (easy to get from date arithmetic) wraps to a positive total_seconds, slips past the if total_seconds < 0 check, and a positive expiry gets persisted from a negative input. one-line fix: let total_seconds = i64::from(delta.get_days()) * 86_400 + i64::from(delta.get_seconds()); then range-check before the as u64.

hubcio · 2026-07-15T09:34:46Z

-        #[gen_stub(override_type(type_repr = "datetime.timedelta | None", imports=("datetime")))]
-        message_expiry: Option<Py<PyDelta>>,
-        #[gen_stub(override_type(type_repr = "builtins.int | None"))] max_topic_size: Option<u64>,
+        #[gen_stub(override_type(type_repr = "IggyExpiry | None"))] message_expiry: Option<


note this is a breaking change for create_topic.message_expiry - it went from timedelta | None to IggyExpiry | None, and that param shipped in the released 0.8.0, so existing callers passing a timedelta now hit a TypeError. the max_topic_size retype and all of update_topic are post-0.8.0 so those break nothing released. worth calling out the message_expiry break, at least in the PR descr.

Sorry, I left a comment but hadn't updated the PR description yet. I know it's sometimes easier to follow a PR when the description reflects the original push and comments show how the discussion evolved, but I should have been clearer here, apologies.

I see three options:

Keep the current change. I think it's the most robust approach, but I understand it's a breaking change for message_expiry.

Revert to the original types used to init the server settings (backwards compatible).

Accept both, support the old type alongside the new enum.

My main concern is whether we should preserve backwards compatibility here and add test coverage for it, but I wanted to get your thoughts before going down that path.

hubcio · 2026-07-15T09:34:46Z

+    /// such by the server. A negative `timedelta` raises `ValueError` when
+    /// this value is passed to `create_topic`/`update_topic`. The upper
+    /// bound is whatever a `datetime.timedelta` can represent that also fits
+    /// in a `u64` microsecond count (about 584,942 years); in practice the


this bound isn't accurate today - the timedelta conversion in py_delta_to_iggy_duration overflows i32 at ~68 years, so the real ceiling is far below what's documented. the figure is also a typo: u64 microseconds is ~584,542 years, not 584,942. once the overflow is widened to i64 the ~584,542y bound becomes real and this doc is correct.

hubcio · 2026-07-15T09:34:46Z

+        Ok(match max_size {
+            MaxTopicSize::ServerDefault() => RustMaxTopicSize::ServerDefault,
+            MaxTopicSize::Custom { bytes } => {
+                if *bytes == 0 || *bytes == u64::MAX {


small consistency thing: MaxTopicSize.Custom(0) (and u64::MAX) reject with ValueError here, but the sibling sentinel IggyExpiry.ExpireDuration(timedelta(0)) silently coerces to server-default instead of rejecting. both are documented so it's not a bug, just worth picking one policy across the two - either both reject the reserved value or both coerce it.

hubcio · 2026-07-15T09:34:46Z

+
+    /// The collection of partitions in the topic.
+    #[getter]
+    pub fn partitions(&self) -> Vec<Partition> {


this getter clones the whole partition vec and builds fresh python objects on every access, so topic.partitions looks like a cheap attribute but re-materializes each time (and the returned objects don't have stable identity). it's a cold metadata path so not a perf concern, but worth a doc note or exposing it as a method so callers don't read it repeatedly in a loop.

hubcio · 2026-07-15T09:34:46Z

@@ -169,7 +170,9 @@ impl IggyClient {
    }

    /// Creates a new topic with the given parameters.


minor: create_topic documents its return and raises as free prose while update_topic below uses the structured Args/Returns/Raises form. worth aligning the two.

hubcio · 2026-07-15T09:34:46Z

-            Some(delta) => IggyExpiry::ExpireDuration(py_delta_to_iggy_duration(&delta)),
-            None => IggyExpiry::ServerDefault,
-        };
+        let expiry = message_expiry


this expiry + max_size resolution block is duplicated byte-for-byte in update_topic. could pull it into a small helper (or two) to dedup, no behavior change.

hubcio · 2026-07-15T09:34:46Z

+        ("max_topic_size_bytes", "expected_exception"),
        [
            (-1, OverflowError),
            (2e64, TypeError),


the create_topic version of this test also covers (0, ValueError) and (2**64-1, ValueError) for the reserved sentinels, but they're missing here even though update goes through the same try_from. there's also no update_topic test for a negative ExpireDuration like create has. worth mirroring the create cases.

Matthew Patton added 2 commits July 6, 2026 19:53

feat(python): expose remaining Topic/TopicDetails fields

f598148

test(python): assert new Topic/TopicDetails fields in topic tests

159c023

mattp5657 marked this pull request as ready for review July 7, 2026 00:01

Merge branch 'master' into feat/python-sdk-topic-details-fields

1c7445c

github-actions Bot added the S-waiting-on-review PR is waiting on a reviewer label Jul 7, 2026

mattp5657 changed the title ~~Feat/python sdk topic details fields~~ feat(python): expose remaining Topic/TopicDetails fields and partitions Jul 7, 2026

slbotbm requested changes Jul 8, 2026

View reviewed changes

github-actions Bot added S-waiting-on-author PR is waiting on author response and removed S-waiting-on-review PR is waiting on a reviewer labels Jul 8, 2026

slbotbm requested changes Jul 8, 2026

View reviewed changes

mattp5657 closed this Jul 9, 2026

github-actions Bot removed the S-waiting-on-author PR is waiting on author response label Jul 9, 2026

mattp5657 reopened this Jul 9, 2026

github-actions Bot added the S-waiting-on-author PR is waiting on author response label Jul 9, 2026

Matthew Patton added 2 commits July 10, 2026 19:16

feat(python): accept IggyExpiry/MaxTopicSize enums in create_topic an…

3758607

…d update_topic

docs(python): spell out IggyExpiry/MaxTopicSize variant semantics

87aabd7

github-actions Bot added S-waiting-on-review PR is waiting on a reviewer and removed S-waiting-on-author PR is waiting on author response labels Jul 10, 2026

slbotbm requested changes Jul 12, 2026

View reviewed changes

github-actions Bot added S-waiting-on-author PR is waiting on author response and removed S-waiting-on-review PR is waiting on a reviewer labels Jul 12, 2026

Matthew Patton added 3 commits July 12, 2026 19:25

fix(python): reject negative timedelta durations

b55642f

docs(python): note ValueError on negative message expiry

78b6500

fix(python): reject max topic size at u64 boundary

a511223

github-actions Bot added S-waiting-on-review PR is waiting on a reviewer and removed S-waiting-on-author PR is waiting on author response labels Jul 12, 2026

hubcio requested changes Jul 15, 2026

View reviewed changes

github-actions Bot added S-waiting-on-author PR is waiting on author response and removed S-waiting-on-review PR is waiting on a reviewer labels Jul 15, 2026

		@@ -169,7 +170,9 @@ impl IggyClient {
		}

		/// Creates a new topic with the given parameters.

Uh oh!

Conversation

mattp5657 commented Jul 7, 2026

Which issue does this PR address?

Rationale

What changed?

Local Execution

AI Usage

Uh oh!

codecov Bot commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

slbotbm left a comment

Choose a reason for hiding this comment

Uh oh!

slbotbm left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

slbotbm commented Jul 9, 2026

Uh oh!

mattp5657 commented Jul 10, 2026

Uh oh!

mattp5657 commented Jul 10, 2026

Uh oh!

mattp5657 commented Jul 10, 2026

Uh oh!

slbotbm left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mattp5657 commented Jul 12, 2026

Uh oh!

mattp5657 commented Jul 12, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mattp5657 Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov Bot commented Jul 7, 2026 •

edited

Loading

mattp5657 Jul 15, 2026 •

edited

Loading