Skip to content

apps: Fix O_ACCMODE bitmask checks after Linux flag alignment#3572

Merged
linguini1 merged 1 commit into
apache:masterfrom
xiaoxiang781216:oflags2
Jun 30, 2026
Merged

apps: Fix O_ACCMODE bitmask checks after Linux flag alignment#3572
linguini1 merged 1 commit into
apache:masterfrom
xiaoxiang781216:oflags2

Conversation

@xiaoxiang781216

Copy link
Copy Markdown
Contributor

Summary

Companion to apache/nuttx#19238 ("include/fcntl.h: align open flags with Linux values").

After aligning the NuttX open() flag constants with Linux (O_RDONLY=0, O_WRONLY=1, O_RDWR=2), code that used (flags & O_RDONLY) or (flags & O_WRONLY) as a bitmask check is broken because O_RDONLY is now 0. This PR fixes all such sites in apps/ by using (flags & O_ACCMODE) comparisons instead.

  • usrsocktest (examples/usrsocktest/): replace flags & O_RDWR with flags & O_ACCMODE in fcntl(F_GETFL) assertions across 5 test files.
  • dd (system/dd/dd_main.c): verify mode used (oflags & O_RDONLY) to detect verify; replace with (oflags & O_ACCMODE) == O_RDWR.
  • dpopen (system/popen/dpopen.c): replace (oflag & O_RDWR) == O_RDWR with (oflag & O_ACCMODE) == O_RDWR.
  • usbmsc (system/usbmsc/usbmsc_main.c): replace flags & O_WRONLY with (flags & O_ACCMODE) == O_RDONLY.
  • fcntl_test (testing/testsuites/kernel/syscall/cases/fcntl_test.c): replace (flags & O_WRONLY) == 0 with (flags & O_ACCMODE) == O_RDONLY.

Impact

  • Correctness: Without these fixes, the affected code silently misbehaves when O_RDONLY == 0 (e.g. dd verify never triggers, dpopen bidirectional detection fails, usbmsc read-only detection is wrong).
  • Dependency: Must be merged together with (or after) !include/fcntl.h: align open flags with Linux values nuttx#19238. Standalone this PR has no effect since apache/nuttx still uses the old flag values.

Testing

Tested on the sim:nsh target (Linux x86_64 host, gcc) together with apache/nuttx#19238.

Boot log:

NuttShell (NSH) NuttX-10.4.0
nsh> uname -a
NuttX 10.4.0 6bb526f459a Jun 29 2026 23:37:06 sim sim
nsh> ps
  TID   PID  PPID PRI POLICY   TYPE    NPX STATE    EVENT     SIGMASK
 STACK COMMAND
    0     0     0   0 FIFO     Kthread   - Ready              0000000000000000 0
069584 Idle_Task
    1     0     0 224 FIFO     Kthread   - Waiting  Semaphore 0000000000000000 0
067456 sim_loop_wq
    2     0     0 224 FIFO     Kthread   - Waiting  Semaphore 0000000000000000 0
067472 hpwork
    4     4     0 100 FIFO     Task      - Running            0000000000000000 0
067496 nsh_main

OSTest: ran ostest from the NSH prompt — all tests passed with no failures or panics.

After aligning NuttX open() flag constants with Linux (O_RDONLY=0,
O_WRONLY=1, O_RDWR=2), code that used '(flags & O_RDONLY)' or
'(flags & O_WRONLY)' as a bitmask check is broken because O_RDONLY
is now 0.  Fix by using '(flags & O_ACCMODE)' comparisons instead.

  - usrsocktest: replace 'flags & O_RDWR' with 'flags & O_ACCMODE'
    in fcntl F_GETFL assertions
  - dd: verify mode used '(oflags & O_RDONLY)' to detect verify;
    replace with '(oflags & O_ACCMODE) == O_RDWR'
  - dpopen: replace '(oflag & O_RDWR) == O_RDWR' with
    '(oflag & O_ACCMODE) == O_RDWR'
  - usbmsc: replace 'flags & O_WRONLY' with
    '(flags & O_ACCMODE) == O_RDONLY'
  - fcntl_test: replace '(flags & O_WRONLY) == 0' with
    '(flags & O_ACCMODE) == O_RDONLY'

Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
@xiaoxiang781216

Copy link
Copy Markdown
Contributor Author

@jerpelea and @linguini1 this patch need merge ASAP to avoid breaking mainline.

@cederom cederom left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @xiaoxiang781216 :-)

@linguini1 linguini1 merged commit 5ead824 into apache:master Jun 30, 2026
78 of 79 checks passed
@xiaoxiang781216 xiaoxiang781216 deleted the oflags2 branch July 1, 2026 02:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants