Maybe we can go with WALBlock?

renamed to WalBlock

fixed

Let's name everything WAL-related consistently. Somewhere you have Wal, somewhere you have WAL.

done

Seems like we could reuse headerBuf on each iteration.
Is it possible to move headerBuf outside of the loop?

done

Maybe we can move these variables to meta_block.go (which must be renamed to wal_block.go in my opinion) and introduce new structure WALHeader with different access methods e.g. Magic() and Version()?

The initial version had exactly that - a dedicated WalHeader struct. But I didn't see much of the point right now since there is no much usage of it.

I wouldn't move constants to wal_block.go since they are not related to the block itself, only to WAL.

Yeah, I see, then a small nit:

const (
	// _WalVersionInitial is the first version of WAL file
	// with CRC32 checksums and 64 byte alignment for blocks.
	_WalVersionInitial byte = iota
)

const (
	// WalMagic is the magic number at the start of WAL files.
	WalMagic uint32 = 0xFFFFFFFF
	// WalCurrentVersion is the current WAL format version.
	WalCurrentVersion = _WalVersionInitial
	// WalHeaderSize is the size of the WAL header in bytes (4 bytes magic + 1 byte version).
	// 59 bytes are also reserved due to alignment.
	WalHeaderSize = 5
)

const (
	// WalBlockAlignment is the alignment boundary for blocks in the new WAL format.
	// Must be greater than [WalBlockHeaderLen] to prevent header torn writes
	// and allow faster navigation during replay of corrupted WAL file.
	WalBlockAlignment int64 = 64
)

That's just my personal preference so you can ignore it (and it's very insignificant, I guess). I've added prefix (_) to _WalVersionInitial to send a signal to other developers that this variable should not be used.

By the way, you use specific WAL version instead of current WAL version in wal_reader.go:

version := header[4]
if version != _WalVersionInitial {
	return nil, fmt.Errorf("unknown WAL version: %d (supported: %d)", version, _WalVersionInitial)
}

Why WALVersion1 and not just WALVersion?

Next line, WALCurrentVersion = WalVersion1

I guess we can get rid of CAS loop where.

Let's rewrite initialization like this:

func NewWalWriter(ws WriteSyncer, offset int64, skipSync bool) *WalWriter {
	w := &WalWriter{
		ws:       ws,
		skipSync: skipSync,
		notify:   make(chan struct{}, 1),
	}

 	// Here we have offset that can be calculated like 64 * `k` for some `k`.
	w.offset.Store(nextBlockOffset(offset))
	// write a header at the beggining if it's a new file
	if offset == 0 {
		if err := writeWALHeader(ws); err != nil {
			logger.Panic("failed to write WAL header", zap.Error(err))
		}

		if !skipSync {
			_ = ws.Sync()
		}


 	    // Here we have offset that can be calculated like 64 * `k` for some `k`.
        // In this case `k` is equal to 1.
		w.offset.Store(nextBlockOffset(WALHeaderSize)) 
	}

	w.wg.Add(1)
	go func() {
		w.syncLoop()
		w.wg.Done()
	}()

	return w
}

So we know that offset is already aligned.
Now we can reserve space like this:

func (w *WalWriter) reserveSpace(blockSize int64) int64 {
	aligned := nextBlockOffset(blockSize)
    
    // w.offset is already aligned.
    // So when we add aligned block we still have aligned offset.
	end := w.offset.Add(aligned)
	start := end - aligned

	return start
}

I've applied these changes locally and tests are green.

yep, if offset is always aligned, then CAS is not needed. fixed

Maybe we can move this to separate method?

done

Why not reusing frac.FileWriter here? Yeah, I've noticed that in FileWriter there is a conversion from new WAL format to old Meta format but I guess it can be done somewhere else...

I do not like the duplication of fsync-coalescing logic here.

That's a fair point. I redesigned a bit - FileWriter now (probably should be named SyncFileWriter) is not aware of any block formats, while convertions of doc => wal happens in a separate wrapper LegacyMetaWriter.

WalWriter now also works on top of FileWriter, which means it now reserves space from FileWriter too, since offset is owned by FileWriter.

Both WalWriter and LegacyMetaWriter implement a trait MetaWriter.

I also moved FileWriter to storage package to prevent circular dependency.

Both WalWriter and LegacyMetaWriter implement a trait

I guess, you've confused languages a little bit 🤣
Overall, looks great.