2024-11-11 - KDB Project Call Meeting Minutes

[alvin-c-shih]

Date

Monday, November 11, 2024 - 9:30am ET / 2:30pm UK

// Second Monday of every month

Attendees

Fullname	Affiliation	GitHub Username
Karl Moll	FINOS	@karlmoll
Peter Gyorok	KX Systems	@gyorokpeter
Aaron Davies	Morgan Stanley	@adavies42
Ross Duffy	Morgan Stanley	@rdffy
Alvin Shih	Morgan Stanley	@alvin-c-shih

Meeting notices

• FINOS Project leads are responsible for observing the FINOS guidelines for running project meetings. Project maintainers can find additional resources in the FINOS Maintainers Cheatsheet.

• All participants in FINOS project meetings are subject to the LF Antitrust Policy, the FINOS Community Code of Conduct and all other FINOS policies.

• FINOS meetings involve participation by industry competitors, and it is the intention of FINOS and the Linux Foundation to conduct all of its activities in accordance with applicable antitrust and competition laws. It is therefore extremely important that attendees adhere to meeting agendas, and be aware of, and not participate in, any activities that are prohibited under applicable US state, federal or foreign antitrust and competition laws. Please contact legal@finos.org with any questions.

• FINOS project meetings may be recorded for use solely by the FINOS team for administration purposes. In very limited instances, and with explicit approval, recordings may be made more widely available.

Agenda

• (5 min) Convene, roll call, welcome new people.

  • Veteran's Day.

• Display FINOS Antitrust Policy summary slide

• Review Meeting Notices (see above)

• (5 min) Approve previous meeting minutes.

  • Approved by acclamation.

• (10 min) kdb 5.0 – The Roadmap Ahead (Ryan Hamilton)

  • Familiar feature requests.
  • A bit unfair to imply Pierre and Oleg not engaging with the community since they attend KX Meetups.
    • Andrew Wilson mentioned as a newer member of the core team.
  • KX Insights ships with pgwire.
    • https://code.kx.com/insights/1.1/core/pgwire.html
    • But s.k dialect of SQL is very limited.
    • An open implementation would make it easier to integrate with custom authentication infrastructure at banks or in the cloud.
  • kdb+ 4.1 has type hints but q linter from KX Analyst needs to be updated to respect those.
    • https://code.kx.com/q/releases/ChangesIn4.1/
    • https://code.kx.com/analyst/libraries/linter/
  • KX Analyst as a standalone IDE might get less use as VS Code ecosystem gets more AI-powered extensions / forks.
  • A standard module system really should be a priority at KX.
    • Few are willing to make the effort to tidy up their code for open source contribution if there are few consumers.
    • And there will be few consumers if there are hurdles.

• (10 min) A First Look at PyKX 3.0 (54 minutes)

  • Includes convenient way to launch the "tick architecture".
    • https://code.kx.com/pykx/3.0/user-guide/advanced/streaming/basic.html
  • Seems to assume a VENV, but a good practice.
  • connection_validator looks useful!
    • Could implement Kerberos ticket validation easily enough.
  • Does PyKX 3.0 generate q as a string?
    • Some forms of access control will reject any kind of lambda.
    • Other plants may expect a "stored proc" call in the form of a general list.
  • ML toolkit now works with PyKX.
    • https://github.com/kxsystems/ml
  • dep.q could work with q modules in VENVs since there are ways to specify the directory that the module lives under.

• (5 min) Repo scraping notebook sketch.

  • https://github.com/alvin-c-shih/kdb/blob/repo-scraping_00/notebooks/repo-scraping/00_q-repo-list.ipynb
  • Searching for topic:q topic:kdb license:apache-2.0 license:mit license:0bsd license:cc only results in 44 repos?
  • Let me know if you come up with a better list for fine-tuning an LLM.

• (10 min) Nominate issues / pull requests for evaluating AI SWEs.

  • Some vendors out there claiming they have agentic AI that can complete a software development task on its own.
  • Easiest if we can point them at public code.
  • Best to have a mix of merged PRs (which has a human-crafted solution to compare against) as well as open issues (in case the AI "cheats").
    • Ideally taken from FINOS projects:
      • https://github.com/finos/qPython
      • https://github.com/finos/kdb-studio
      • https://github.com/finos/kdb
    • Willing to consider others if they're a Big Win for the kdb+ community.
  • Ideally code you're familiar with and can assign a difficulty rating.

• (5 min) AOB, Q&A, Adjourn.

Join Zoom Meeting

• https://zoom.us/j/94523415326?pwd=NkJDaUtUUlZTcHVwcUJHdG0yRFJUdz09
• Meeting ID: 945 2341 5326
• Passcode: 361990
• Find your local number: https://zoom.us/u/abTtYRZ0B6

[karlmoll]

Karl Moll / FINOS

[rdffy]

Ross Duffy / Morgan Stanley

[adavies42]

Aaron Davies/Morgan Stanley

[gyorokpeter]

Péter Györök / KX

[adavies42]

searching with the term in quotes, https://github.com/search?q=language%3A%22q%22&type=repositories, gets only things labelled exactly "q", and matches 825 repos

[alvin-c-shih]

Alvin Shih / Morgan Stanley

[cmccarthy1]

Hey!

Apologies all that none of the PyKX team had been on to discuss the various items today (Rian was off on leave and I'm currently flying to NY). On the items raised the demo script for PyKX 3.0 first look was using an older version of the jupyter exit functionality, there's a wrapped function in the pykx.q script

.pykx.disableJupyter[]

Which will call the above line under the hood. Open to additional suggestions though! I'll look to be on next months call as we'll have released 3.0 by then and can have a conversation on the remaining/other items then! Happy to have a separate session if needed.