logtail: the deferred counter remembers a place in the log, not a second (stage 2b)
test / test (push) Canceled after 0s
test / test (push) Canceled after 0s
The counter passed over, while catching up after a start, every warning no later than the last one counted - by time; mail.log stamps whole seconds, so a second warning of that second, later in the file, was lost. - The tailer gives each line its place (Place: the file's fingerprint and the offset just past the line): follow tracks the bytes the reader hands over, readRotated likewise in mail.log.1. Run hands tr.line to the counter. - store.AddSenderCheckDeferred(at, n, place) counts, moves the last-deferral mark (never back - an upsert WHERE) and saves the place (settings sender_check_counted_place) in one transaction. - Catching up, a warning in the file of the last one counted is passed over by place (end <= counted end); in another file - a rotated log read again - by time, as before. The fingerprint carries the inode, so the place stays true across a rotation. Tests: two warnings of one second, an unclean stop between them, all lines read again - each counted once; the store saves the place with the count and keeps the latest deferral time. Existing counter tests pass a zero place (time rule). CHANGELOG [Unreleased], audit plan line ticked, roadmap 24/62. Evidence: go vet ./... clean; go test ./internal/store, ./internal/logtail ok (but for the Windows-only test); the server run follows before the push. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -46,6 +46,34 @@ type tracker struct {
|
||||
// no longer the log - rotated while the panel was down; prev is that place.
|
||||
behind bool
|
||||
prev store.LogtailState
|
||||
|
||||
// line is the place of the line being handled: set by follow and
|
||||
// readRotated before each call of handle.
|
||||
line Place
|
||||
}
|
||||
|
||||
// Place is where a line of the log ends: the fingerprint of its file and the
|
||||
// byte offset just past it. A rotated file keeps its fingerprint (the inode
|
||||
// is part of it), so a place stays true across a rotation.
|
||||
type Place struct {
|
||||
FP string
|
||||
End int64
|
||||
}
|
||||
|
||||
// String is the place as it is saved: "<fingerprint>@<offset>".
|
||||
func (p Place) String() string { return fmt.Sprintf("%s@%d", p.FP, p.End) }
|
||||
|
||||
// parsePlace reads a saved place; the zero Place for anything else.
|
||||
func parsePlace(s string) Place {
|
||||
i := strings.LastIndexByte(s, '@')
|
||||
if i < 0 {
|
||||
return Place{}
|
||||
}
|
||||
var end int64
|
||||
if _, err := fmt.Sscan(s[i+1:], &end); err != nil {
|
||||
return Place{}
|
||||
}
|
||||
return Place{FP: s[:i], End: end}
|
||||
}
|
||||
|
||||
// resume returns the byte offset the tailer should start reading f at, having
|
||||
@@ -125,10 +153,14 @@ func (t *tracker) readRotated(handle func(string)) bool {
|
||||
lost(fmt.Sprintf("seek %s: %v", rotated, err))
|
||||
return false
|
||||
}
|
||||
fp := fingerprintOf(f)
|
||||
end := t.prev.Offset
|
||||
r := bufio.NewReader(f)
|
||||
for {
|
||||
line, err := r.ReadString('\n')
|
||||
end += int64(len(line))
|
||||
if line != "" {
|
||||
t.line = Place{FP: fp, End: end}
|
||||
handle(strings.TrimRight(line, "\r\n"))
|
||||
}
|
||||
if err != nil {
|
||||
|
||||
Reference in New Issue
Block a user